Quantization

inference_models.Quantization

Bases: str, Enum

Attributes

BF16 class-attribute instance-attribute

BF16 = 'bf16'

FP16 class-attribute instance-attribute

FP16 = 'fp16'

FP32 class-attribute instance-attribute

FP32 = 'fp32'

INT8 class-attribute instance-attribute

INT8 = 'int8'

UNKNOWN class-attribute instance-attribute

UNKNOWN = 'unknown'