g? ddlZddlZddlmZddlZddlmZddlZdZdZ dejfdZ d*dejd e d eejd eejfd Zej j"d dfdejdejdej de def dZdej(fdZdej(fdZdej(fdZdej(dede fdZdej(dede gfdZdejfdZded ed!ej:d"ej:fd#Zdej(ded efd$Zdej(fd%Z dejBfd&Z"d'Z#Gd(d)Z$y)+N)Optional)version_converterc |jD]E}||fi||jD]+}|jdst|j|fi|-Gy)a Iterate the graph including subgraphs calling the per_node_func for each node. :param graph: Graph to iterate :param per_node_func: Function to call for each node. Signature is fn(node: onnx:NodeProto, **kwargs) :param func_args: The keyword args to pass through. gN)node attributeHasFielditerate_graph_per_node_funcr)graph per_node_func func_argsrattrs W/var/www/openai/venv/lib/python3.12/site-packages/onnxruntime/tools/onnx_model_utils.pyr r sL d(i(NND}}S!+DFFMOYO#c ||fi||jD]<}|jD]+}|jdst|j|fi|->y)a# Iterate the graph including subgraphs calling the per_graph_func for each Graph. :param graph: Graph to iterate :param per_graph_func: Function to call for each graph. Signature is fn(graph: onnx:GraphProto, **kwargs) :param func_args: The keyword args to pass through. rN)rrr iterate_graph_per_graph_funcr)r per_graph_funcr rrs rrrsL5&I& NND}}S!,TVV^QyQ#rmodelcji}|jD]!}|jxsd}|j||<#|S)zu Get the opsets imported by the model :param model: Model to check. :return: Map of domain to opset. zai.onnx) opset_importdomainversion)ropsetsentryrs rget_opsets_importedr/s; F##*v$ Mr model_pathopsetout_pathloggerc&t|jd}|r|jd||tj|}t j ||}|r3tj|t||r|jd||S)a Helper to update the opset of a model using onnx version_converter. Target opset must be greater than current opset. :param model_path: Path to model to update :param opset: Opset to update model to :param out_path: Optional output path for updated model to be saved to. :param logger: Optional logger for diagnostic output :returns: Updated onnx.ModelProto TstrictzUpdating %s to opset %dzSaved updated model to %s)strresolveinfoonnxloadrconvert_versionsave)rrrrmodel_path_strr new_models rupdate_onnx_opsetr,>s}++4+89N  -~uE IIn %E!11%?I )S]+  KK3X > rF output_pathlevel log_leveluse_external_initializersctj}t|j|_||_||_|rltjjdd\}}}t|t|fdk\r|jddntdtjtjt|jd|d g } y ) a Optimize an ONNX model using ONNX Runtime to the specified level :param model_path: Path to ONNX model :param output_path: Path to save optimized model to. :param level: onnxruntime.GraphOptimizationLevel to use. Default is ORT_ENABLE_BASIC. :param log_level: Log level. Defaults to Error (3) so we don't get output about unused initializers being removed. Warning (2) or Info (1) may be desirable in some scenarios. :param use_external_initializers: Set flag to write initializers to an external file. Required if model > 2GB. Requires onnxruntime 1.17+ .r-)z7session.optimized_model_external_initializers_file_namezexternal_data.pbzONNX Runtime 1.17 or higher required to save initializers as external data when optimizing model. Current ONNX Runtime version is Tr!CPUExecutionProvider) providersN) ortSessionOptionsr#r$optimized_model_filepathgraph_optimization_levellog_severity_level __version__splitintadd_session_config_entry ValueErrorInferenceSession) rr.r/r0r1somajorminorrest_s roptimize_modelrH]s"   B"%k&9&9&;".update_dim_valuessmBww .++11$yy<< 4JZ9ZIIK,1CM ) rinputoutput value_info)r kwargsrVrUrKs @@r_replace_symbolic_dim_valuer\sDk* 7OE2ekk"ell#e&&'rcd}|jD] }|| |jD] }|| |jD] }|| y)Nc|jjdrg|jjj}|rD|jD]4}|jds|j dks%|j 6yyy)NrMrRr4)rNr rMrOrPrRrQ)rKrOrPs rclear_invalid_valuesz=_remove_invalid_dim_values_impl..clear_invalid_valuessd ::  } -JJ**00E 99C||K0S]]Q5F % .rrW)r r_iorTs r_remove_invalid_dim_values_implrbsQ$[[Q\\QR rc$t|ty)z Iterate the graph and subgraphs, unsetting any dim_value entries that have a value of less than 1. These are typically erroneously inserted by a converter to represent a dynamic dimension. :param graph: GraphProto to update N)rrb)r s rremove_invalid_dim_valuesrds !(GHr param_namerKc*t|t||y)z Iterate all values in the graph, replacing dim_param in a tensor shape with the provided value. :param graph: GraphProto to update :param param_name: dim_param to set :param value: value to use )rJrKN)rr\)r rerKs rmake_dim_param_fixedrgs!(Czafgr input_name fixed_shapec rt||jD]V}|j|k(s|jj dst d|d|jj j}t|jt|k7r-t dt|jdt|t|jD]\}}|j dr7|j||k7s*t d|jd||d |d z|j d rt||j||z|j|||_ y t d|d dj|jDcgc]}|jc}cc}w)a Update the named graph input to set shape to the provided value. This can be used to set unknown dims as well as to replace dim values. If setting the input shape replaces a dim_param, update any other values in the graph that use the dim_param. :param graph: Graph to update :param input_name: Name of graph input to update. :param fixed_shape: Shape to use. rMzInput z is not a tensorzRank mismatch. Existing:z Replacement:rRz%Can't replace existing fixed size of z with z for dimension r4rJNz7 was not found in graph inputs. Valid input names are: ,)rdrXnamerNr rArMrOlenrP enumeraterRrgrJrQjoin)r rhrir`rOidxrPs rmake_input_shape_fixedrqse$ [[ 66Z 66??=1 6*5E!FGGFF&&,,E599~[!11 #;C N;K=Y\]hYiXj!kll%eii0S<< ,}} C(88(CCMM?RXYdehYiXjk--01WI7\\+.( {3?OPIIK$/$4CM1 58   ""%((EKK+HKqAFFK+H"I!J L +HsF4ctjj|}tjj |t |j jD]\}}t|r|j j|}t|s7|jjjj|jjjy)a  Update the output shapesof a model where the input shape/s were made fixed, if possible. This is mainly to make the model usage clearer if the output shapes can be inferred from the new input shapes. :param model: Model that had input shapes fixed. N) r&shape_inference infer_shapeschecker check_modelrnr rYis_fixed_size_tensorrNrMrOCopyFrom)rm2rpranew_os rfix_output_shapesr{s    * *5 1BLLR EKK../Q#A&HHOOC(E#E*""((11%**2H2H2N2NO 0rnode_to_producersnode_to_consumersproducerconsumerc||vr t||<||vr t||<||j|||j|y)a Create links between two nodes for a value produced by one and consumed by the other. :param node_to_producers: Map of NodeProto to set of nodes that produce values the node consumes as inputs. :param node_to_consumers: Map of NodeProto to set of nodes that consume values the node produces as outputs. :param producer: Producer node :param consumer: Consumer node N)setadd)r|r}r~rs r_create_producer_consumer_linkrsX((&)e(#((&)e(#h##H-h##H-rcV |jDchc]}|jc} |jDchc]}|jc} it} fd}|jD]}|jDcgc]}|}}|j D]0}|j dst|j||} || z }2|D]7}|s||r|vs|} t||| |'|j|9|jD]} || < |Scc}wcc}wcc}w)Nc$|vxs |vxs|vSN)rK graph_inputs initializers producerss ris_local_valuez._map_node_dependencies..is_local_values" !SUl%:Se|>SSrr) rXrl initializerrrrr _map_node_dependenciesrrrrY)r r|r}r`implicit_inputsrrinputsrsubgraph_implicit_inputsr~rarrrs @@@rrrs.$)KK0KqAFFK0L$)$5$56$5qAFF$56LIeOT !ZZ(Z!Z(NND}}S!+A$&&J[]n+o(22# Aa  >(|H23DFWYacgh##A&AIaL). E16)sDD!< D&cdtj_i}i}t|||}|r&t ddj t |||fS)ab Get maps for connections between the node that produces each value and the nodes that consume the value. Processing includes subgraphs. As the map key is a Node instance from the Graph there should be no ambiguity. :param graph: Graph to process. :return: Tuple with two maps. First is node_to_producers map of a node to set of all nodes producing input it consumes. Second is node_to_consumers map of a node to set of all nodes consuming output it creates. e.g. NodeA and NodeB provide inputs to NodeC. NodeC provides input to NodeD node_to_consumers[NodeA] = set([NodeC]) node_to_consumers[NodeB] = set([NodeC]) node_to_producers[NodeC] = set([NodeA, NodeB]) node_to_consumers[NodeC] = set([NodeD]) node_to_producers[NodeD] = set([NodeC]) ct|Sr)idselfs rz,get_producer_consumer_maps..Js2d8rz;This appears to be an invalid model with missing inputs of rk)r& NodeProto__hash__rrArosorted)r r|r}rs rget_producer_consumer_mapsr8sg$4DNN,U4EGXYOI#((SYZiSjJkIl m   / //rcd}|jjdrZ|jjj}|r8d}|jD]'}|jdr|j dkDr$d}|S|S)z Check if value is a tensor with a fixed shape. :param value: onnx.ValueInfoProto to check :return: True if value is a tensor, with a shape, where all dimensions have fixed values. FrMTrRr)rNr rMrOrPrR)rKis_fixedrOrPs rrwrwZsvH zz=) &&,, Hyy<< ,1B! O! Orc|dk(rtjjS|dk(rtjjS|dk(rtjjS|dk(rtjj St d|z)z)Convert string to GraphOptimizationLevel.disablebasicextendedallzInvalid optimization level of )r8GraphOptimizationLevelORT_DISABLE_ALLORT_ENABLE_BASICORT_ENABLE_EXTENDEDORT_ENABLE_ALLrA)r/s rget_optimization_levelrqsz ))999 ))::: ))=== ~))888 5= >>rc6eZdZdZdej fdZdZy)ModelProtoWithShapeInfoac Class to load an ONNX model and run shape inferencing on it to populate the ValueInfo. The model_with_shape_info property will contain the updated model. If the model is > 2GB and uses external data a temporary file is required to run shape inferencing successfully. This helper class handles automatic removal of the temporary file. rcp||_tjt|}tjj |d|_d|_t|jjdkDrt|j jjdk(rtj|jd|_tjjt|t|jdtjt|j|_yyy)z] :param model_path: Path to ONNX model to load and run shape inferencing on. T) strict_modeNrz.temp_with_shapeinf.onnx)rr&r'r#rsrtmodel_with_shape_info_tmp_model_pathrmr rpathlibPath with_suffixinfer_shapes_path)rrrs r__init__z ModelProtoWithShapeInfo.__init__s % #j/*%)%9%9%F%FuZ^%F%_" $ u{{ 1 $T-G-G-M-M-R-R)SWX)X#*<< #;#G#GHb#cD  2 23z?CH\H\D]ko 2 p)-3t7K7K3L)MD &*Y $rcV|jr|jjdyy)NT) missing_ok)runlinkrs r__del__zModelProtoWithShapeInfo.__del__s'     ' '4 ' 8 rN)__name__ __module__ __qualname____doc__rrrrrrrrrsN7<<N$9rr)NN)%loggingrtypingrr&r onnxruntimer8r r ModelProtorrr?Loggerr,rrboolrH GraphProtor\rbrdr#rgrqr{dictrrrrValueInfoProtorwrrrrrrs " P R" t $(,'+   w||$ W^^ $ D),(B(B(S(S&+ "k "k"k  % %"k "k $ "kJ(t(&!4??!&IT__IhhShh,$//,s,RUQV,^PT__P$..04.@D.Z^ZhZh.,#$//#d#_c#L0doo0D 3 3. ? 99r