Skip to content

Commit bedb7c8

Browse files
committed
Add TorchDynamo fp32 inference examples (resnet50 & BERT)
1 parent 462604a commit bedb7c8

File tree

3 files changed

+79
-0
lines changed

3 files changed

+79
-0
lines changed

docs/tutorials/examples.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -294,6 +294,52 @@ with torch.no_grad():
294294
model(data)
295295
```
296296

297+
#### TorchDynamo Mode (Experimental, _NEW feature from 2.0.0_)
298+
299+
##### Resnet50
300+
301+
```
302+
import torch
303+
import torchvision.models as models
304+
305+
model = models.resnet50(weights=models.ResNet50_Weights.DEFAULT)
306+
model.eval()
307+
data = torch.rand(1, 3, 224, 224)
308+
309+
# Experimental Feature
310+
#################### code changes ####################
311+
import intel_extension_for_pytorch as ipex
312+
model = torch.compile(model, backend="ipex")
313+
######################################################
314+
315+
with torch.no_grad():
316+
model(data)
317+
```
318+
319+
##### BERT
320+
321+
```
322+
import torch
323+
from transformers import BertModel
324+
325+
model = BertModel.from_pretrained("bert-base-uncased")
326+
model.eval()
327+
328+
vocab_size = model.config.vocab_size
329+
batch_size = 1
330+
seq_length = 512
331+
data = torch.randint(vocab_size, size=[batch_size, seq_length])
332+
333+
# Experimental Feature
334+
#################### code changes ####################
335+
import intel_extension_for_pytorch as ipex
336+
model = torch.compile(model, backend="ipex")
337+
######################################################
338+
339+
with torch.no_grad():
340+
model(data)
341+
```
342+
297343
### BFloat16
298344

299345
Similar to running with FP32, the `optimize` function also works for BFloat16 data type. The only difference is setting `dtype` parameter to `torch.bfloat16`.
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
import torch
2+
from transformers import BertModel
3+
4+
model = BertModel.from_pretrained("bert-base-uncased")
5+
model.eval()
6+
7+
vocab_size = model.config.vocab_size
8+
batch_size = 1
9+
seq_length = 512
10+
data = torch.randint(vocab_size, size=[batch_size, seq_length])
11+
12+
#################### code changes ####################
13+
import intel_extension_for_pytorch as ipex
14+
model = torch.compile(model, backend="ipex")
15+
######################################################
16+
17+
with torch.no_grad():
18+
model(data)
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
2+
import torch
3+
import torchvision.models as models
4+
5+
model = models.resnet50(weights=models.ResNet50_Weights.DEFAULT)
6+
model.eval()
7+
data = torch.rand(1, 3, 224, 224)
8+
9+
#################### code changes ####################
10+
import intel_extension_for_pytorch as ipex
11+
model = torch.compile(model, backend="ipex")
12+
######################################################
13+
14+
with torch.no_grad():
15+
model(data)

0 commit comments

Comments
 (0)