Skip to content

Commit 128a48b

Browse files
authored
Update readme.md
1 parent c85ed50 commit 128a48b

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

docs/neuron-cc/readme.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ The Neuron Compiler is an Ahead-of-Time (AoT) compiler that accepts Machine Lear
44

55
The Neuron compiler analyzes the user-provided compute-graph, and performs various optimizations such as loop-fusion, tensorization, scheduling, and memory management, which significantly improves inference throughput and memory usage.
66

7-
AoT compilation requires that dynamic tensor shapes (dimension sizes) of all tensors in the compute-graph are known at compilation time, in order for the compiler to make sound decisions. If any shape cannot be determined at compile time.
7+
AoT compilation requires that dynamic tensor shapes (dimension sizes) of all tensors in the compute-graph are known at compilation time, in order for the compiler to make sound decisions. If any shape cannot be determined at compile time compilation will fail.
88

99
It is common for developers to train in FP32, for avoiding the challenges of low-precision training (e.g. loss-scaling, etc). However, during inference, developers typically look for the most cost-effective target. In order to address these two requirements, Neuron supports auto-conversion, which takes FP32 models as input, and then runs them at speed of 16-bit using BFloat16 model, using our FP32 to BF16 auto conversion.
1010
The Neuron compiler is most often used within an integrated framework, such as Tensorflow-Neuron. From that framework,

0 commit comments

Comments
 (0)