gbdZddlZddlZddlZddlZddlmZmZddlm Z ddl m Z ddl m Z mZmZmZmZmZmZmZmZmZddlmZdd lmZmZd d lmZmZmZej@e!Z"e rddl#Z#dd edd d dd dddee$e fdee$de%dee&e$fdeee$e$fde%de%deee$fdZ'dd edd d dd dee$dfdee$e fdee$de%dee&e$fdeee$e$fde%de%deee$ddfdZ(ejReddee$dfde$dee&e$fdefdZ*dd ddddd ddd!ee$ejVfd"e%d#e%d$e%d%eee$d&fd'e%dee$defd(Z,ddejRd)dddejVd"e%d$e%de$def d*Z- dHd+ee$ejVfd%eee$d&fd$e%d'e%deee$dfeff d,Z.ddd-ee$ddfd.Z/d/ddee&eed0fffd1Z0d/ddeed&ee&eed0ffe&ffd2Z1d/dde&fd3Z2e dId4Z3d/ddee&eed0fffd5Z4 dJdee$dfdee$e$fde%deee$fd6Z5d/dde&fd7Z6d8eee$dee$dfdeee$fd9Z7dee$dfdeee$fd:Z8d/dde%fd;Z9ddd<dee$dfd=eee$d>eee$dee$ee$ffd?Z:e d@dAde&fdBZ;GdCdDedEdFdGgZ If one of the model's tensor is bigger than `max_shard_size`, it will end up in its own shard which will have a size greater than `max_shard_size`. If your model is a `transformers.PreTrainedModel`, you should pass `model._tied_weights_keys` as `shared_tensors_to_discard` to properly handle shared tensors saving. This ensures the correct duplicate tensors are discarded during saving. Args: model (`torch.nn.Module`): The model to save on disk. save_directory (`str` or `Path`): The directory in which the model will be saved. filename_pattern (`str`, *optional*): The pattern to generate the files names in which the model will be saved. Pattern must be a string that can be formatted with `filename_pattern.format(suffix=...)` and must contain the keyword `suffix` Defaults to `"model{suffix}.safetensors"` or `pytorch_model{suffix}.bin` depending on `safe_serialization` parameter. force_contiguous (`boolean`, *optional*): Forcing the state_dict to be saved as contiguous tensors. This has no effect on the correctness of the model, but it could potentially change performance if the layout of the tensor was chosen specifically for that reason. Defaults to `True`. max_shard_size (`int` or `str`, *optional*): The maximum size of each shard, in bytes. Defaults to 5GB. metadata (`Dict[str, str]`, *optional*): Extra information to save along with the model. Some metadata will be added for each dropped tensors. This information will not be enough to recover the entire shared structure but might help understanding things. safe_serialization (`bool`, *optional*): Whether to save as safetensors, which is the default behavior. If `False`, the shards are saved as pickle. Safe serialization is recommended for security reasons. Saving as pickle is deprecated and will be removed in a future version. is_main_process (`bool`, *optional*): Whether the process calling this is the main process or not. Useful when in distributed training like TPUs and need to call this function from all processes. In this case, set `is_main_process=True` only on the main process to avoid race conditions. Defaults to True. shared_tensors_to_discard (`List[str]`, *optional*): List of tensor names to drop when saving shared tensors. If not provided and shared tensors are detected, it will drop the first name alphabetically. Example: ```py >>> from huggingface_hub import save_torch_model >>> model = ... # A PyTorch model # Save state dict to "path/to/folder". The model will be split into shards of 5GB each and saved as safetensors. >>> save_torch_model(model, "path/to/folder") # Load model back >>> from huggingface_hub import load_torch_model # TODO >>> load_torch_model(model, "path/to/folder") >>> ``` ) state_dictrrrrrr!rrN)save_torch_state_dictr#) r r!rrrrrrrs Y/var/www/openai/venv/lib/python3.12/site-packages/huggingface_hub/serialization/_torch.pysave_torch_modelr&'s1n##%))%-%'"; r#z torch.Tensorreturnc t|}|"|rtjntj}|i}|r ddlm} t||||}nddlm } tjdt|||} |rtj|jd d z} t!j"|D]a} | j%| s tj'd | d t!j(t j*j-|| cddi}| j0s|j3||rd|ini}| j4j7D]X\} }|Dcic]}||| }}| |t j*j-|| fi|tj'd| Z| j0r|jd dz}i| j8|| j:d}t=t j*j-||d5}t?j@||ddddtjCd|dtE| j4d|dtjCd|d y#t $r} t d| d} ~ wwxYw#t.$r'} tjd| d| dYd} ~ d} ~ wwxYwcc}w#1swYxYw)!a Save a model state dictionary to the disk, handling sharding and shared tensors issues. See also [`save_torch_model`] to directly save a PyTorch model. For more information about tensor sharing, check out [this guide](https://huggingface.co/docs/safetensors/torch_shared_tensors). The model state dictionary is split into shards so that each shard is smaller than a given size. The shards are saved in the `save_directory` with the given `filename_pattern`. If the model is too big to fit in a single shard, an index file is saved in the `save_directory` to indicate where each tensor is saved. This helper uses [`split_torch_state_dict_into_shards`] under the hood. If `safe_serialization` is `True`, the shards are saved as safetensors (the default). Otherwise, the shards are saved as pickle. Before saving the model, the `save_directory` is cleaned from any previous shard files. If one of the model's tensor is bigger than `max_shard_size`, it will end up in its own shard which will have a size greater than `max_shard_size`. If your model is a `transformers.PreTrainedModel`, you should pass `model._tied_weights_keys` as `shared_tensors_to_discard` to properly handle shared tensors saving. This ensures the correct duplicate tensors are discarded during saving. Args: state_dict (`Dict[str, torch.Tensor]`): The state dictionary to save. save_directory (`str` or `Path`): The directory in which the model will be saved. filename_pattern (`str`, *optional*): The pattern to generate the files names in which the model will be saved. Pattern must be a string that can be formatted with `filename_pattern.format(suffix=...)` and must contain the keyword `suffix` Defaults to `"model{suffix}.safetensors"` or `pytorch_model{suffix}.bin` depending on `safe_serialization` parameter. force_contiguous (`boolean`, *optional*): Forcing the state_dict to be saved as contiguous tensors. This has no effect on the correctness of the model, but it could potentially change performance if the layout of the tensor was chosen specifically for that reason. Defaults to `True`. max_shard_size (`int` or `str`, *optional*): The maximum size of each shard, in bytes. Defaults to 5GB. metadata (`Dict[str, str]`, *optional*): Extra information to save along with the model. Some metadata will be added for each dropped tensors. This information will not be enough to recover the entire shared structure but might help understanding things. safe_serialization (`bool`, *optional*): Whether to save as safetensors, which is the default behavior. If `False`, the shards are saved as pickle. Safe serialization is recommended for security reasons. Saving as pickle is deprecated and will be removed in a future version. is_main_process (`bool`, *optional*): Whether the process calling this is the main process or not. Useful when in distributed training like TPUs and need to call this function from all processes. In this case, set `is_main_process=True` only on the main process to avoid race conditions. Defaults to True. shared_tensors_to_discard (`List[str]`, *optional*): List of tensor names to drop when saving shared tensors. If not provided and shared tensors are detected, it will drop the first name alphabetically. Example: ```py >>> from huggingface_hub import save_torch_state_dict >>> model = ... # A PyTorch model # Save state dict to "path/to/folder". The model will be split into shards of 5GB each and saved as safetensors. >>> state_dict = model_to_save.state_dict() >>> save_torch_state_dict(state_dict, "path/to/folder") ``` Nr) save_filezjPlease install `safetensors` to use safe serialization. You can install it with `pip install safetensors`.)rr)saveaYou are using unsafe serialization. Due to security reasons, it is recommended not to load pickled models from untrusted sources. If you intend to share your model, we strongly recommend using safe serialization by installing `safetensors` with `pip install safetensors`.rrz(-\d{5}-of-\d{5})?suffixz(\.index\.json)?zRemoving existing file 'z' from folder.z&Error when trying to remove existing 'z' from folder: z. Continuing...formatptrzShard saved to .index.json)r weight_mapwr)indentz:The model is bigger than the maximum size per checkpoint (z#). Model weighs have been saved in z^ checkpoint shards. You can find where each parameters has been saved in the index located at .z$Model weights successfully saved to !)#strr SAFETENSORS_WEIGHTS_FILE_PATTERNPYTORCH_WEIGHTS_FILE_PATTERNsafetensors.torchr* ImportError!_clean_state_dict_for_safetensorstorchr+loggerwarning"split_torch_state_dict_into_shardsrecompiler/oslistdirmatchdebugremovepathjoin Exception is_shardedupdatefilename_to_tensorsitemsrtensor_to_filenameopenjsondumpinfolen)r#r!rrrrrrr save_file_fnestate_dict_splitexisting_files_regexfilenameper_file_metadatasafe_file_kwargstensorstensorshard index_pathindexfs r%r$r$sf(N"  6 677    C7  -&?   / c :%5n !zz*:*A*AI^*A*_bu*uv >2H#))(3LL#;H:^!TUIIbggll>8DE 3"4(  & &  *:L $56RT-AAGGI':AB'F++'BUBGGLLBWFVW xj12J ""%,,B,7-G A+44AA*== "'',,~z:C @A IIeQq )A HHXY//23C3W3W/X.YZYYcXdde g  KK6~6FaHI E  B!NN@ /Z[Y\\klCA @sBJ A J)& KK! J& J!!J&) K2KK!K*r,c2t|||ttS)a> Split a model state dictionary in shards so that each shard is smaller than a given size. The shards are determined by iterating through the `state_dict` in the order of its keys. There is no optimization made to make each shard as close as possible to the maximum size passed. For example, if the limit is 10GB and we have tensors of sizes [6GB, 6GB, 2GB, 6GB, 2GB, 2GB] they will get sharded as [6GB], [6+2GB], [6+2+2GB] and not [6+2+2GB], [6+2GB], [6GB]. To save a model state dictionary to the disk, see [`save_torch_state_dict`]. This helper uses `split_torch_state_dict_into_shards` under the hood. If one of the model's tensor is bigger than `max_shard_size`, it will end up in its own shard which will have a size greater than `max_shard_size`. Args: state_dict (`Dict[str, torch.Tensor]`): The state dictionary to save. filename_pattern (`str`, *optional*): The pattern to generate the files names in which the model will be saved. Pattern must be a string that can be formatted with `filename_pattern.format(suffix=...)` and must contain the keyword `suffix` Defaults to `"model{suffix}.safetensors"`. max_shard_size (`int` or `str`, *optional*): The maximum size of each shard, in bytes. Defaults to 5GB. Returns: [`StateDictSplit`]: A `StateDictSplit` object containing the shards and the index to retrieve them. Example: ```py >>> import json >>> import os >>> from safetensors.torch import save_file as safe_save_file >>> from huggingface_hub import split_torch_state_dict_into_shards >>> def save_state_dict(state_dict: Dict[str, torch.Tensor], save_directory: str): ... state_dict_split = split_torch_state_dict_into_shards(state_dict) ... for filename, tensors in state_dict_split.filename_to_tensors.items(): ... shard = {tensor: state_dict[tensor] for tensor in tensors} ... safe_save_file( ... shard, ... os.path.join(save_directory, filename), ... metadata={"format": "pt"}, ... ) ... if state_dict_split.is_sharded: ... index = { ... "metadata": state_dict_split.metadata, ... "weight_map": state_dict_split.tensor_to_filename, ... } ... with open(os.path.join(save_directory, "model.safetensors.index.json"), "w") as f: ... f.write(json.dumps(index, indent=2)) ``` )rrget_storage_sizeget_storage_id)rget_torch_storage_sizeget_torch_storage_id)r#rrs r%rArA.s!F 0%)/+  r'F)strictsafe weights_only map_locationmmaprcheckpoint_pathrhrirjrkz torch.devicerlct|}|jstd|d|jr!t |||}|j ||S|Jt j}||jddzz } | js|st j}||jddzz } | jrt|||||St|j|rd nd } t| d k(r%t | d ||| }|j ||Std|d)a Load a checkpoint into a model, handling both sharded and non-sharded checkpoints. Args: model (`torch.nn.Module`): The model in which to load the checkpoint. checkpoint_path (`str` or `os.PathLike`): Path to either the checkpoint file or directory containing the checkpoint(s). strict (`bool`, *optional*, defaults to `False`): Whether to strictly enforce that the keys in the model state dict match the keys in the checkpoint. safe (`bool`, *optional*, defaults to `True`): If `safe` is True, the safetensors files will be loaded. If `safe` is False, the function will first attempt to load safetensors files if they are available, otherwise it will fall back to loading pickle files. `filename_pattern` parameter takes precedence over `safe` parameter. weights_only (`bool`, *optional*, defaults to `False`): If True, only loads the model weights without optimizer states and other metadata. Only supported in PyTorch >= 1.13. map_location (`str` or `torch.device`, *optional*): A `torch.device` object, string or a dict specifying how to remap storage locations. It indicates the location where all tensors should be loaded. mmap (`bool`, *optional*, defaults to `False`): Whether to use memory-mapped file loading. Memory mapping can improve loading performance for large models in PyTorch >= 2.1.0 with zipfile-based checkpoints. filename_pattern (`str`, *optional*): The pattern to look for the index file. Pattern must be a string that can be formatted with `filename_pattern.format(suffix=...)` and must contain the keyword `suffix` Defaults to `"model{suffix}.safetensors"`. Returns: `NamedTuple`: A named tuple with `missing_keys` and `unexpected_keys` fields. - `missing_keys` is a list of str containing the missing keys, i.e. keys that are in the model but not in the checkpoint. - `unexpected_keys` is a list of str containing the unexpected keys, i.e. keys that are in the checkpoint but not in the model. Raises: [`FileNotFoundError`](https://docs.python.org/3/library/exceptions.html#FileNotFoundError) If the checkpoint file or directory does not exist. [`ImportError`](https://docs.python.org/3/library/exceptions.html#ImportError) If safetensors or torch is not installed when trying to load a .safetensors file or a PyTorch checkpoint respectively. [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError) If the checkpoint path is invalid or if the checkpoint format cannot be determined. Example: ```python >>> from huggingface_hub import load_torch_model >>> model = ... # A PyTorch model >>> load_torch_model(model, "path/to/checkpoint") ``` zCheckpoint path z does not exist)checkpoint_filerkrjrhr1r-r2)r r!rhrjrz *.safetensorsz*.binrr)rorkrjrlz Directory 'zw' does not contain a valid checkpoint. Expected either a sharded checkpoint with an index file, or a single model file.)rexists ValueErroris_fileload_state_dict_from_fileload_state_dictrr9r/r:_load_sharded_checkpointlistglobrU) r rmrhrirjrkrlrr#r` model_filess r%load_torch_modelrz}sut?+O  ! ! #+O+= 1.13. filename_pattern (`str`, *optional*, defaults to `"model{suffix}.safetensors"`): The pattern to look for the index file. Pattern must be a string that can be formatted with `filename_pattern.format(suffix=...)` and must contain the keyword `suffix` Defaults to `"model{suffix}.safetensors"`. Returns: `NamedTuple`: A named tuple with `missing_keys` and `unexpected_keys` fields, - `missing_keys` is a list of str containing the missing keys - `unexpected_keys` is a list of str containing the unexpected keys r1r-r2rzutf-8)encodingNr3cpu)rkrjrp) missing_keysunexpected_keys)r/rDrIrJrQrRload!_validate_keys_for_strict_loadingkeysrwsetvaluesrtrur#_IncompatibleKeys)r r!rhrjrr` index_filerbra shard_files shard_file shard_pathr# loaded_keys model_keyss r%rvrvs<H"(((3mCJnj9J j# 0A !  1 )%|1D1I1I1KLs5.55789K! WW\\.*= . % j8 "eL)..01KU%%',,./J *{23T+XbJbEc 7 1 0s EE rocXt|}|jstd|d|jdk(r ddlm}ddlm}||d 5}|j} ddd "| jd d vrtd |d|!t|drt|jn|} | dk(rtj!dd} ||| S ddl} ddlm} i} t'j(| j*t'j(dk\r|| d<t'j(| j*t'j(dk\r|| d<| |fd|i| S#t$r}td|d}~wwxYw#1swY"xYw#t$r}td|d}~wwxYw)aW Loads a checkpoint file, handling both safetensors and pickle checkpoint formats. Args: checkpoint_file (`str` or `os.PathLike`): Path to the checkpoint file to load. Can be either a safetensors or pickle (`.bin`) checkpoint. map_location (`str` or `torch.device`, *optional*): A `torch.device` object, string or a dict specifying how to remap storage locations. It indicates the location where all tensors should be loaded. weights_only (`bool`, *optional*, defaults to `False`): If True, only loads the model weights without optimizer states and other metadata. Only supported for pickle (`.bin`) checkpoints with PyTorch >= 1.13. Has no effect when loading safetensors files. mmap (`bool`, *optional*, defaults to `False`): Whether to use memory-mapped file loading. Memory mapping can improve loading performance for large models in PyTorch >= 2.1.0 with zipfile-based checkpoints. Has no effect when loading safetensors files, as the `safetensors` library uses memory mapping by default. Returns: `Union[Dict[str, "torch.Tensor"], Any]`: The loaded checkpoint. - For safetensors files: always returns a dictionary mapping parameter names to tensors. - For pickle files: returns any Python object that was pickled (commonly a state dict, but could be an entire model, optimizer state, or any other Python object). Raises: [`FileNotFoundError`](https://docs.python.org/3/library/exceptions.html#FileNotFoundError) If the checkpoint file does not exist. [`ImportError`](https://docs.python.org/3/library/exceptions.html#ImportError) If safetensors or torch is not installed when trying to load a .safetensors file or a PyTorch checkpoint respectively. [`OSError`](https://docs.python.org/3/library/exceptions.html#OSError) If the checkpoint file format is invalid or if git-lfs files are not properly downloaded. [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError) If the checkpoint file path is empty or invalid. Example: ```python >>> from huggingface_hub import load_state_dict_from_file # Load a PyTorch checkpoint >>> state_dict = load_state_dict_from_file("path/to/model.bin", map_location="cpu") >>> model.load_state_dict(state_dict) # Load a safetensors checkpoint >>> state_dict = load_state_dict_from_file("path/to/model.safetensors") >>> model.load_state_dict(state_dict) ``` zNo checkpoint file found at 'zO'. Please verify the path is correct and the file has been properly downloaded.z .safetensorsr) safe_open) load_filezoPlease install `safetensors` to load safetensors checkpoint. You can install it with `pip install safetensors`.Nr0) frameworkr/)r0mlxz"The safetensors archive passed at zg does not contain the valid metadata. Make sure you save your model with the `save_torch_model` method.typemetazJMeta device is not supported with safetensors. Falling back to CPU device.r~)device)rzZPlease install `torch` to load torch tensors. You can install it with `pip install torch`.z2.1.0rlz1.13.0rjrk)rrsFileNotFoundErrorr. safetensorsrr;rr<rgetOSErrorhasattrr8rr?r@r>rrparse __version__)rorkrjrlrmrrrWrbrrr>radditional_kwargss r%rtrt.sj?+O  " " $+O+<=5 5  /  - 3$ 71zz|H8  HLL$:-$O4_4EFJJ ,8+CP\^dHe\&&'kw V  NNg hF88 }}U&&'7==+AA$(&!}}U&&'7==+BB,8.)  !   M E  8 7$  k  s; E%F F% E?. E::E?F  F) F$$F)rct|}t|jj}||z }||z }|s|rd|jj}|r5dj t |Dcgc]}d|d c}}|d|dz }|r5dj t |Dcgc]}d|d c}} |d| dz }t|ycc}wcc}w)a Validate that model keys match loaded keys when strict loading is enabled. Args: model: The PyTorch model being loaded loaded_keys: The keys present in the checkpoint Raises: RuntimeError: If there are missing or unexpected keys in strict mode z#Error(s) in loading state_dict for ,"z Missing key(s): r6z Unexpected key(s): N)rr#r __class____name__rJsorted RuntimeError) r rloaded_keys_setrrr error_messagekstr_missing_keysstr_unexpected_keyss r%rrs+&OU%%',,./J/L% 2O=eoo>V>V=WX "xx6,;O(P;Oa1QCq;O(PQ  12B1C1E EM "%((f_>U+V>Uas!H>U+V"W  45H4IK KM=))')Q,Ws 4 C+ Cr^.c2 ddlm}|r'j\}}tfd|DS j j dk(r+tr!ddl}|jj}|St}|S#t$rY\wxYw)zReturns a unique id for plain tensor or a (potentially nested) Tuple of unique id for the flattened Tensor if the input is a wrapper tensor subclass Tensor ris_traceable_wrapper_subclassc3HK|]}tt|ywN)_get_unique_idgetattr.0attrr^s r% z!_get_unique_id..sQ54(=>5"xlaN) torch.utils._python_dispatchr__tensor_flatten__tupler<rris_torch_tpu_available torch_xla_XLAC_xla_get_tensor_id storage_ptr)r^rattrs_r unique_ids` r%rrs  N ( 0002HE1Q5QQ Q 1}}U"'='? OO66v>   '     s4B BBcx|jjdk(ry|jt|t|fS)a, Return unique identifier to a tensor storage. Multiple different tensors can share the same underlying storage. This identifier is guaranteed to be unique and constant for this tensor's storage during its lifetime. Two tensor storages with non-overlapping lifetimes may have the same id. In the case of meta tensors, we return None since we can't tell if they share the same storage. Taken from https://github.com/huggingface/transformers/blob/1ecf5f7c982d761b4daaa96719d162c324187c64/src/transformers/pytorch_utils.py#L278. rN)rrrrf)r^s r%rgrgs5}}V#}}nV46LV6TTTr'c ddlm}|r'j\}}tfd|DS j j S#t$rY*wxYw#t$ro jjtjzcYS#t$r+jtjzcYcYSwxYwwxYw)z Taken from https://github.com/huggingface/safetensors/blob/08db34094e9e59e2f9218f2df133b7b4aaff5a99/bindings/python/py_src/safetensors/torch.py#L31C1-L41C59 rrc3HK|]}tt|ywr)rfrrs r%rz)get_torch_storage_size..s!WQV-gfd.CDQVr)rrrsumr<untyped_storagenbytesAttributeErrorstoragesize_get_dtype_sizedtypeNotImplementedErrornelementr^rrrs` r%rfrfs  N ( 0002HE1WQVWW W 1 E%%'..00    E E>>#((*_V\\-JJ J" E??$v||'DD D E Es@4AA' A$#A$' C13B'$C'/CCCCctjjd|r ddlmcm}|j }yyy#t$rYywxYw)z Checks if `torch_xla` is installed and potentially if a TPU is in the environment Taken from https://github.com/huggingface/transformers/blob/1ecf5f7c982d761b4daaa96719d162c324187c64/src/transformers/utils/import_utils.py#L463. rNrTF) importlibutil find_spectorch_xla.core.xla_modelcore xla_model xla_devicer) check_devicexmrs r%rr sQ~~ ,8  55MMO    s? A  A c ddlm}||r t|S |j j S#t$rY*wxYw#t $r2 |jj cYS#t$rYYywxYwwxYw)z Taken from https://github.com/huggingface/safetensors/blob/079781fd0dc455ba0fe851e2b4507c33d0c0d407/bindings/python/py_src/safetensors/torch.py#L11. rr) rrrr<rdata_ptrrKrr)r^rs r%rr s N ( 0!&) ) 1 %%'0022     >>#,,. ."   s>;A AA BA41B4 B=BBBct||}|jD]\}}|D]}|i}||vr|||<||=|r1|jD cic]\}} || j}}} |Scc} }w)a6Remove shared tensors from state_dict and update metadata accordingly (for reloading). Warning: `state_dict` and `metadata` are mutated in-place! Taken from https://github.com/huggingface/safetensors/blob/079781fd0dc455ba0fe851e2b4507c33d0c0d407/bindings/python/py_src/safetensors/torch.py#L155. ) discard_names)_remove_duplicate_namesrO contiguous) r#rrr to_removes kept_nameto_remove_group to_removervs r%r=r=9s)C\]J&0&6&6&8" ?(I(&/#9%)'94>4D4D4FG4FDAqa'4F G HsA2c|jr:|jddjt|jz}|S|j}|S)z Taken from https://github.com/huggingface/safetensors/blob/079781fd0dc455ba0fe851e2b4507c33d0c0d407/bindings/python/py_src/safetensors/torch.py#L23. )rviewrrr)r^stops r%_end_ptrrTsQ{{2r"++- 0MM K  Kr'r]cg}|D]}t|dkr|j|#g}|D]2}||}|j|jt||f4|j |d\}}} |j| h|ddD]4\} } }| |k\r|j|hn|dj || }6|S)z Taken from https://github.com/huggingface/safetensors/blob/079781fd0dc455ba0fe851e2b4507c33d0c0d407/bindings/python/py_src/safetensors/torch.py#L44 rrrNr)rUappendrrsortadd) r]r#filtered_tensorssharedareasnamer^r last_stop last_namestartrs r%_filter_shared_not_sharedr_s v;?  # #F + D%F LL&//+Xf-=tD E  "'(9i ,!&qr E4 ! ''/ $((.I "+( r'cddl}tt}|jD]v\}}|j|jdk7s&t |dk7s5t |dk7sD||jt |t |fj|xtt|j}t||}|S)z Taken from https://github.com/huggingface/safetensors/blob/079781fd0dc455ba0fe851e2b4507c33d0c0d407/bindings/python/py_src/safetensors/torch.py#L69. rNr) r>rrrOrrrfrrwrrr)r#r> tensors_dictrrr]s r%_find_shared_tensorsr{ss#L  "1 88|u||F+ + A!0CH^_`HaefHf !((KN4J14MN O S STU V#6,--/01G'.sMut|GFD$9:ur) rrrallr<rrrrrrfrs` r%rrs  N ( 0002HE1MuMM M 1 ??  F 3 3 (8IO M9  '9((    s4B BB)preferred_namesrrrc .|g}t|}|g}t|}t|}tt}|D]}t|Dcgc]}t ||s|c}} | st d|dt t| d} | j|} | rt t| d} |r*|j| } | rt t| d} t |D]}|| k7s || j||Scc}w)rzvError while trying to find names to remove to save state dict, but found no suitable name to keep for saving amongst: z. None is covering the entire storage. Refusing to save/load the model since you could be storing much more memory than needed. Please refer to https://huggingface.co/docs/safetensors/torch_shared_tensors for more information. Or open an issue.r) rrrrwrrr difference intersectionr) r#rrunique_preferred_namesunique_discard_namessharedsrrrcomplete_names keep_name preferreds r%rrs7 1 }-":.GD!IvXvtjQUFV9WdvXY((.x0 4/03 #--.BC tI/2I !.;;NKI"4 ?3A6 6NDy )$++D1#38 7Ys D D rz torch.dtypecDddl}t|dd}t|dd}|jd|jd|jd|j d|j d|jd|jd|jd|jd|jd|d|di }||S) z Taken from https://github.com/huggingface/safetensors/blob/08db34094e9e59e2f9218f2df133b7b4aaff5a99/bindings/python/py_src/safetensors/torch.py#L344 rN float8_e4m3fn float8_e5m2rr) r>rint64float32int32bfloat16float16int16uint8int8boolfloat64)rr>_float8_e4m3fn _float8_e5m2_SIZEs r%rrs UOT:N5-6L Q q Q  q Q Q A A qa E <r'c,eZdZdZdeffd ZeZxZS)rz This is used to report missing and unexpected keys in the state dict. Taken from https://github.com/pytorch/pytorch/blob/main/torch/nn/modules/module.py#L52. r(cR|js |jsyt| S)Nz)rrsuper__repr__)selfrs r%rz_IncompatibleKeys.__repr__s%  )=)=4w!!r')r __module__ __qualname____doc__r8r__str__ __classcell__)rs@r%rrs "#" Gr'rIncompatibleKeysrr)NFF)T)TN)=r rrRrDrB collectionsrr functoolsrpathlibrtypingrrr r r r r rrr packagingrr1rr_baserrr get_logger__file__r?r>r8rintr&r$r9rAPathLikerzrvrtrrrgrfrrr=rrrrrrrr'r%r/s) /ddd!WW   H %'+!&4)-# 59a a#t)$asm a  a #s(O atCH~&aaa (S 2aP'+!&4)-# 59`JS.()`J#t)$`Jsm `J  `J #s(O `JtCH~&`J`J`J (S 2`J `JL&FF&4 IS.()II#s(O I  If9=&*h h3 +,h  h  h  h5n!456h hsmhh^%FF C CKKC  C  C  CCP:> m3 +,m5n!456mm  m  4^# $c )* mf* *#* *<>eCsCx4H.I@UUHU>SXY\^cdgildl^mYmSnpsCs=t4uU"E>EcE6  (5eCHo1E+F8"59 S.()38n (S 2 6^tCH~4^H[C\aefijmfnao8 T#~*=%> 4C>  ((D(.,0)- .S.().d3i(.DI& .  #tCy. .b =S 4  #5HY7Z[ r'