g\ddlZddlmZmZddlZej eZGddZy)N)ListTupleceZdZdZed defdZedZedZede e e jfdZ ed de e jd efd Z edde e e jfd Zy )PastKeyValuesHelperzEHelper functions to process past key values for encoder-decoder modelpresentcg}g}t|D]L}|j|r d|d|gn d|d|g|j|r d|d|gn d|d|gN||zS) Npresent_key_self_present_value_self_past_key_self_past_value_self_present_key_cross_present_value_cross_past_key_cross_past_value_cross_)rangeextend) num_layersrpast_self_namespast_cross_namesis c/var/www/openai/venv/lib/python3.12/site-packages/onnxruntime/transformers/models/t5/past_helper.pyget_past_namesz"PastKeyValuesHelper.get_past_namessz"A  " "%QC(,?s*CD&qc*.>qc,BC   # #&aS)-A!+EF's+/@-DE  #!111cg}g}t|D]S\}}t|dk(sJdt||\}}}}|j||g|j||gU||fS)aSplit present state from grouped by layer to grouped by self/cross attention. Before: (past_key_self_0, past_value_self_0, past_key_cross_0, past_value_cross_0), (past_key_self_1, past_value_self_1, past_key_cross_1, past_value_cross_1), ... After: (past_key_self_0, past_value_self_0, past_key_self_1, past_value_self_1, ...), (past_key_cross_0, past_value_cross_0, past_key_cross_1, past_value_cross_1, ...) !Expected to have four items. Got  enumeratelenr) present_key_values present_self present_cross_ipresent_layer_ipresent_key_selfpresent_value_selfpresent_key_crosspresent_value_crosss rgroup_by_self_or_crossz*PastKeyValuesHelper.group_by_self_or_cross#s  #,-?#@ B'1, h0QRUVeRfQg.h h,    "!#   !13E F G  "35H!I J$A]**rchtdzk(sJtfdtDS)aReorder past state from grouped by self/cross attention to grouped by layer. Before: past_key_self_0, past_value_self_0, past_key_self_1, past_value_self_1, ..., past_key_cross_0, past_value_cross_0, past_key_cross_1, past_value_cross_1, ... After: (past_key_self_0, past_value_self_0, past_key_cross_0, past_value_cross_0), (past_key_self_1, past_value_self_1, past_key_cross_1, past_value_cross_1), rc3~K|]4}d|zd|zdzdzd|zzdzd|zzdzg6yw)N).0rrpasts r z5PastKeyValuesHelper.group_by_layer..?sh ' QU QUQYQ^a!e+,Q^a!e+a/0   's:=)rtupler)r0rs``rgroup_by_layerz"PastKeyValuesHelper.group_by_layer8s< 4yA N*** :&   rpast_key_valuescd}t|dz}tt|dzD])}d|z}|||||dz|||z|||zdzffz }+|S)aCategorize present_key_values from self and cross attention to layer by layer. Reorder past state from grouped by self/cross attention to grouped by layer. Before: past_key_self_0, past_value_self_0, past_key_self_1, past_value_self_1, ..., past_key_cross_0, past_value_cross_0, past_key_cross_1, past_value_cross_1, ... After: (past_key_self_0, past_value_self_0, past_key_cross_0, past_value_cross_0), (past_key_self_1, past_value_self_1, past_key_cross_1, past_value_cross_1), Args: present_key_values: From past_key_values of a model (group by self and cross attention) Returns: past_tuples: present key and values grouped by layer. r.r,rr-)rr)r4 past_tupleshalf_idxridxs rback_group_by_layerz'PastKeyValuesHelper.back_group_by_layerIs  '1,s?+q01Aa%C #C(#C!G,#HsN3#HsNQ$67  K2rr concatcg}g}t|D]S\}}t|dk(sJdt||\}}}} |j||g|j|| gU|r||zS||fS)aCategorize present_key_values into self and cross attention. Split present state from grouped by layer to grouped by self/cross attention. Before: (past_key_self_0, past_value_self_0, past_key_cross_0, past_value_cross_0), (past_key_self_1, past_value_self_1, past_key_cross_1, past_value_cross_1), ... After: (past_key_self_0, past_value_self_0, past_key_self_1, past_value_self_1, ...), (past_key_cross_0, past_value_cross_0, past_key_cross_1, past_value_cross_1, ...) Args: present_key_values: From past_key_values of a model (group by layer) concat: If concat self attention with cross attention key/value to return Returns: present_self (Tuple[torch.Tensor]): present key and values from self attention present_cross (Tuple[torch.Tensor]): present key and values from cross attention rrr) r r:r!r"_r$r%r&r'r(s rgroup_by_self_and_crossz+PastKeyValuesHelper.group_by_self_and_crossgs$,. ,. "+,>"? A'1, h0QRUVeRfQg.h h,[j X 02CEX   !13E F G  "35H!I J #@ -/ /. .rcHg}|rt|dzn t|}|sdnd}t|D],}|jd|d|fDcgc]}||z c}.t|D],}|jd|d|fDcgc]}||z c}.|Scc}wcc}w)zProcess input names of model wrapper. Args: past_key_values: Consider `self` and `cross` past_key_values Returns: names (List[string]): input names rpast_present_ key_self_ value_self_ key_cross_ value_cross_)rrr)r4encodernamesrprefixrss rget_input_namesz#PastKeyValuesHelper.get_input_namess29S)Q.s??S 'Zz"A LL1#+aS@Q.RS.R&1*.RS T#z"A LLA3/?rVs.   8 $GGr