gjddlZddlZddlmZddlmZmZddlmZm Z m Z m Z m Z e Ze Ze dZeGddZeGdd ZeGd d Zy) N) defaultdict) dataclassfield)DictListLiteralOptionalTuple) F64F32F16BF16I64I32I16I8U8BOOLcdeZdZUdZeed<eeed<eeefed<e dZ eed<d d Z y) TensorInfoa<Information about a tensor. For more details regarding the safetensors format, check out https://huggingface.co/docs/safetensors/index#format. Attributes: dtype (`str`): The data type of the tensor ("F64", "F32", "F16", "BF16", "I64", "I32", "I16", "I8", "U8", "BOOL"). shape (`List[int]`): The shape of the tensor. data_offsets (`Tuple[int, int]`): The offsets of the data in the file as a tuple `[BEGIN, END]`. parameter_count (`int`): The number of parameters in the tensor. dtypeshape data_offsetsFinitparameter_countNc tjtj|j|_y#t $r d|_YywxYw)N) functoolsreduceoperatormulrr TypeError)selfs W/var/www/openai/venv/lib/python3.12/site-packages/huggingface_hub/utils/_safetensors.py __post_init__zTensorInfo.__post_init__#s; %#,#3#3HLL$**#MD  %#$D  %s36A A returnN) __name__ __module__ __qualname____doc__DTYPE_T__annotations__rintr rrr&r%rr s:  N 9S/! e,OS,%r1rcheZdZUdZeeefed<eeefed<e dZ ee e fed<d dZ y) SafetensorsFileMetadataaMetadata for a Safetensors file hosted on the Hub. This class is returned by [`parse_safetensors_file_metadata`]. For more details regarding the safetensors format, check out https://huggingface.co/docs/safetensors/index#format. Attributes: metadata (`Dict`): The metadata contained in the file. tensors (`Dict[str, TensorInfo]`): A map of all tensors. Keys are tensor names and values are information about the corresponding tensor, as a [`TensorInfo`] object. parameter_count (`Dict[str, int]`): A map of the number of parameters per data type. Keys are data types and values are the number of parameters of that data type. metadatatensorsFrrNctt}|jjD]#}||jxx|j z cc<%t ||_yN)rr/r5valuesrrdict)r$rtensors r%r&z%SafetensorsFileMetadata.__post_init__BsJ.9#.>ll))+F FLL )V-C-C C ),#O4r1r')r)r*r+r,rstrr. TENSOR_NAME_Trrrr-r/r&r0r1r%r3r3+sD"38n -+ ,,*/U*;OT'3,';5r1r3ceZdZUdZeeed<eed<eee fed<ee e fed<e dZ ee efed<d d Zy ) SafetensorsRepoMetadataa Metadata for a Safetensors repo. A repo is considered to be a Safetensors repo if it contains either a 'model.safetensors' weight file (non-shared model) or a 'model.safetensors.index.json' index file (sharded model) at its root. This class is returned by [`get_safetensors_metadata`]. For more details regarding the safetensors format, check out https://huggingface.co/docs/safetensors/index#format. Attributes: metadata (`Dict`, *optional*): The metadata contained in the 'model.safetensors.index.json' file, if it exists. Only populated for sharded models. sharded (`bool`): Whether the repo contains a sharded model or not. weight_map (`Dict[str, str]`): A map of all weights. Keys are tensor names and values are filenames of the files containing the tensors. files_metadata (`Dict[str, SafetensorsFileMetadata]`): A map of all files metadata. Keys are filenames and values are the metadata of the corresponding file, as a [`SafetensorsFileMetadata`] object. parameter_count (`Dict[str, int]`): A map of the number of parameters per data type. Keys are data types and values are the number of parameters of that data type. r4sharded weight_mapfiles_metadataFrrNctt}|jjD]1}|jj D]\}}||xx|z cc<3t ||_yr7)rr/rAr8ritemsr9)r$r file_metadatarnb_parameters_s r%r&z%SafetensorsRepoMetadata.__post_init__jsc.9#.>!00779M)6)F)F)L)L)N%~&.8&*O: $O4r1r')r)r*r+r,r rr.boolr< FILENAME_Tr3rrr-r/r&r0r1r%r>r>IsV2tn M]J.//%<<==*/U*;OT'3,';5r1r>)rr! collectionsr dataclassesrrtypingrrrr r r;rGr<r-rr3r>r0r1r%rKsy#(77  V W %% %: 55 5: %5%5 %5r1