g}G ddlmZddlZddlZddlmZddlmZddlZ ddl Z ddl m Z m Z ddlmZddlmZdd lmZd d lmZej,ej.hZej2ej4hZej8ej:hZd hZd Z ddZ!e jDej4ej4dddddddf ddZ#GddZ$y)) annotationsN)Path)Any)CalibrationDataReaderCalibrationMethod) QuantType)StaticQuantConfig)TensorQuantOverridesHelper)'MixedPrecisionTensorQuantOverridesFixerCastlcztjd|d|jd|d|jd|d| y)NzUnable to override z for z node's z because it has already been overridden! Check the initial quantization overrides provided to get_qnn_qdq_config() if the generated QDQ model does not run on QNN EP. Node name: z, z name: )loggingwarningop_typename)nodewhat_str tensor_nameio_kinds r/var/www/openai/venv/lib/python3.12/site-packages/onnxruntime/quantization/execution_providers/qnn/quant_config.pywarn_unable_to_overridersJ  OO hZuT\\N(7)Lii[7)7;- AFTc z| "|tjtjhv} t|tj r|nt j |d} t} d}i}| jjD]3}|||j<tjj|s2d}5t|rtj|ni}|j!s+|r)t#j$|| |}|j'||t)|||| |||}| jj*D].}| j-|j.|j1|0dd| |j3|| | d}t5d| j6D}|j8dkrJt:j=t>tAfd|jCD}|vs|vs|rd|d <tE||||tG| jItJ||xs| jMtNk\| S) a Returns a static quantization configuration suitable for running QDQ models on QNN EP. This is done primarily by setting tensor-level quantization overrides. Params: model_input: Path to the input model file or ModelProto. calibration_data_reader: Calibration data reader. calibrate_methode: The calibration method. Defaults to MinMax. activation_type: The default activation quantization type. Defaults to QUInt8. weight_type: The default weight quantization type. Defaults to QUInt8. per_channel: Global option that determines if a fixed set of operator types should be quantized per-channel. Defaults to false. Alternatively, use the tensor-level `init_overrides` to select individual operators and their quantization axes. If set, the quantization tool uses per-channel quantization for the following operator types and inputs: - Conv: - input[1] on axis 0 - input[2] (bias) on axis 0 - ConvTranspose: - input[1] on axis 1 - input[2] (bias) on axis 0 init_overrides: Initial tensor-level quantization overrides. Defaults to None. This function updates of a copy of these overrides with any necessary adjustments and includes them in the returned configuration object (i.e., config.extra_options['TensorQuantOverrides']). The key is a tensor name and the value is a list of dictionaries. For per-tensor quantization, the list contains a single dictionary. For per-channel quantization, the list contains either a dictionary for each channel in the tensor or a single dictionary that is assumed to apply to all channels. An 'axis' key must be present in the first dictionary for per-channel quantization. Each dictionary contains optional overrides with the following keys and values. 'quant_type' = QuantType : The tensor's quantization data type. 'axis' = Int : The per-channel axis. Must be present for per-channel weights. 'scale' = Float : The scale value to use. Must also specify `zero_point` if set. 'zero_point' = Int : The zero-point value to use. Must also specify `scale` is set. 'symmetric' = Bool : If the tensor should use symmetric quantization. Invalid if also set `scale` or `zero_point`. 'reduce_range' = Bool : If the quantization range should be reduced. Invalid if also set `scale` or `zero_point`. Only valid for initializers. 'rmax' = Float : Override the maximum real tensor value in calibration data. Invalid if also set `scale` or `zero_point`. 'rmin' = Float : Override the minimum real tensor value in calibration data. Invalid if also set `scale` or `zero_point`. 'convert' = Dict : A nested dictionary with the same keys for an activation tensor that should be converted to another quantization type. 'convert["recv_nodes"] = Set : Set of node names that consume the converted activation, other nodes get the original type. If not specified, assume all consumer nodes get the converted type. add_qtype_converts: True if this function should automatically add "convert" entries to the provided `init_overrides` to ensure that operators use valid input/output types (activations only). Ex: if you override the output of an Add to 16-bit, this option ensures that the activation inputs of the Add are also up-converted to 16-bit and that data types for surrounding ops are converted appropriately. Refer to the documentation in mixed_precision_overrides_utils.py for additional details. activation_symmetric: True if activations should be quantized symmetrically (i.e, rmax == -rmin) by default. Defaults to false. For int8 and int16, this results in zero-point values of 0. For uint8 and uin16, the zero-point values are 128 and 32,768, respectively. weight_symmetric: True if weights should be quantized symmetrically (i.e., rmax == -rmin) by default. Defaults to None. If set to None, weight_symmetric is assumed true if the weight_type is a signed int. keep_removable_activations: Defaults to false. If true, "removable" activations (e.g., Clip or Relu) will not be removed, and will be explicitly represented in the QDQ model. If false, these activations are automatically removed if activations are asymmetrically quantized. Keeping these activations is necessary if optimizations or EP transformations will later remove QuantizeLinear/DequantizeLinear operators from the model. Returns: A StaticQuantConfig object F)load_external_dataTg-C6?)MinimumRealRangeDedicatedQDQPairQDQKeepRemovableActivationsTensorQuantOverridesActivationSymmetricWeightSymmetricCalibStridedMinMaxc3^K|]%}|jdk(s|jdk(s"|'yw)zai.onnxN)domain).0xs r z%get_qnn_qdq_config..s)_!3Aqxx2~U^I^a!3s#--c3&K|]}|v ywN)r't opset21_typess rr)z%get_qnn_qdq_config..s*jGi!1 +=GisUseQDQContribOps)calibrate_methodactivation_type weight_typeop_types_to_quantize per_channeluse_external_data_format extra_options)(r QInt8QInt16 isinstanceonnx ModelProto load_modelsetgraph initializerrexternal_data_helperuses_external_datar copydeepcopyemptyr create_from_modelapplyQnnCompatibilityOverridesraddr process_nodeget_dictnext opset_importversion Q16_TYPESunionQ4_TYPESanyget_quant_typesr list differenceOP_TYPES_TO_EXCLUDEByteSizeMODEL_SIZE_THRESHOLD) model_inputcalibration_data_readerr1r2r3r5init_overridesadd_qtype_convertsactivation_symmetricweight_symmetrickeep_removable_activationsstridemodelop_typesmodel_has_external_dataname_to_initializerr@overrides_helperoverrides_fixer qnn_compatrr7 onnx_opsetoverrides_have_opset21_typesr/s @rget_qnn_qdq_configrj+s)b&9??Idtyperscale zero_point?Nrrsget_node_output_qtype_infooutputrqrr QUInt16rnparrayfloat32uint16r9int16rzr output_types rrwz*QnnCompatibilityOverrides._process_sigmoidMs ||y(U,Ft||n*UU(nn?? KKND99 *  )++ + NN 2 2 A"-XXm2::F"$((1BII"> I,, , NN 2 2 A"-XXm2::F"$((1BHH"= -rc |jdk(sJd|j|jj|jd|jj }|t jk(rt|jj|jd|tjdtjtjdtjdy|t jk(rt|jj|jd|tjdtjtjdtjdyy) z^ Overrides 16-bit Tanh's output scale and zero-point as per QNN requirements. rpzExpected Tanh, but got rrrirNrrs rrxz'QnnCompatibilityOverrides._process_tanhis ||v%O)@'OO%nn?? KKND99 *  )++ + NN 2 2 A"-XXm2::F"$((5 "B I,, , NN 2 2 A"-XXm2::F"$((1BHH"= -rN)rqr rrr r]boolr^rr5rrsr rtzdict[str, onnx.TensorProto])ronnx.NodeProto) __name__ __module__ __qualname____doc__r{rJrrurvrwrxr-rrrHrHsp  "+ ( #      . 2 2 5in&.8rrH)rrrstrrrrr)rYzstr | Path | onnx.ModelProtorZrr1rr2r r3r r5rr[z&dict[str, list[dict[str, Any]]] | Noner\rr]rr^z bool | Noner_rr`z int | Nonereturnr )% __future__rrCrpathlibrtypingrnumpyrr; calibraterr quant_utilsr quantizer tensor_quant_overridesr mixed_precision_overrides_utilsr r9rrOr8QUInt8rQInt4QUInt4rQrVrXrMinMaxrjrHr-rrrsp #  A$)@T   y00 1 OOY-- . OOY-- .h!      "+<*B*B!*!1!1&--=A#!&$(',Y-Y2Y(Y Y  Y  Y;YYY"Y!%Y YYx||r