gvL"ddlmZddlZddlZddlZddlmZddlm Z ddl m Z ddl m Z mZmZmZmZddlmZmZddlmZmZmZmZmZdd lmZdd lmZdd lm Z m!Z!m"Z"dd l#m$Z$e!jJe&Z'dZ(er ddl)Z(d e$de$fdZ-d/dZ.dZ/dZ0 d0de de1dee2fdZ3e- d1dee4e fdeee4e fde1de1deee5e4ff dZ6d2dZ7e"e-dd ddddddddddddd!d"e4dee2d#e4d$ee1d%ee4d&ee4d'ee4d(ee1d)eeee4e4fd*eeee4e4fd+eeee4e4fd,ee4de1deee5e4fde1fd-Z8Gd.deZ9y#e*$rddl+Z,e,jPZ(YwxYw)3Nwraps)Path)copytree)AnyDictListOptionalUnion) ModelHubMixinsnapshot_download)get_tf_versionis_graphviz_availableis_pydot_availableis_tf_available yaml_dump) constants)HfApi)SoftTemporaryDirectoryloggingvalidate_hf_hub_args) CallableTfnreturnc.tfd}|S)Ncft|dstdjd|g|i|S)Nhistoryz Cannot use 'z}': Keras 3.x is not supported. Please save models manually and upload them using `upload_folder` or `huggingface-cli upload`.)hasattrNotImplementedError__name__)modelargskwargsrs P/var/www/openai/venv/lib/python3.12/site-packages/huggingface_hub/keras_mixin.py_innerz'_requires_keras_2_model.._inner+sHui(%r{{m,rr %)$)&))r)rr&s` r%_requires_keras_2_modelr()s  2Y** Mr'cg}|jD]g\}}|r|d|n|}t|tjr*|j t ||jU|j ||fit|S)aFlatten a nested dictionary. Reference: https://stackoverflow.com/a/6027615/10319735 Args: dictionary (`dict`): The nested dictionary to be flattened. parent_key (`str`): The parent key to be prefixed to the children keys. Necessary for recursing over the nested dictionary. Returns: The flattened dictionary. .)items isinstance collectionsMutableMappingextend _flatten_dictappenddict) dictionary parent_keyr+keyvaluenew_keys r%r0r07s E &&( U+5ZL#'3 e[77 8 LL%'   LL'5) *) ;r'cd}|jv|jj}t|}tjj j |d<d}|jD]\}}|d|d|dz }|S)z6Parse hyperparameter dictionary into a markdown table.Ntraining_precisionz*| Hyperparameters | Value | | :-- | :-- | z| z | z | ) optimizer get_configr0kerasmixed_precision global_policynamer+)r"tableoptimizer_paramsr5r6s r%_create_hyperparameter_tablerBTs E " ??557()9:161F1F1T1T1V1[1[-.>*002JC r#c%- -E3 Lr'c Ztjj||dddddddd y)N /model.pngFTTB`)to_file show_shapes show_dtypeshow_layer_namesrankdir expand_nesteddpi layer_range)r<utils plot_model)r"save_directorys r% _plot_networkrRbs< KK !"*-  r'Trepo_dirrPmetadatac|dz }|jryt|}|r trtr t |||i}d|d<d}|t |dz }|dz }|dz }|d z }|d z }||d z }|d z }|d z }||z }|dz }|rAt jj|dr|dz }|dz }|dz }d}|d|dz }|dz }|j|y)zd Creates a model card for the repository. Do not overwrite an existing README.md file. z README.mdNr< library_namez--- F)default_flow_stylez/ ## Model description More information needed z9 ## Intended uses & limitations More information needed z: ## Training and evaluation data More information needed z ## Training procedure z ### Training hyperparameters z; The following hyperparameters were used during training:  rDz ## Model Plot z
z$ View Model Plot z ./model.pngz ![Model Image](z) z
) existsrBrrrRrospath write_text)r"rSrPrT readme_pathhyperparameters model_card path_to_plots r%_create_model_cardraps;[(K259O+-2D2FeX&&H^J)H??J'JGGJQQJRRJ"11 88 VV o% d bggnnz%<=** m# >> $ ),s;; n$ :&r'FrQconfiginclude_optimizertagsc *t td|js tdt |}|j dd|rit |tstdt|d|tjz jd5}tj||dddi}t |tr||d <nt |t r|g|d <|j#d d} | 9t%j&d t(d |vr|d j+| n| g|d <|j,|j,j,ik7rx|d z } | j/rt%j&d t0| jdd5}tj|j,j,|dddddt3||||tj4j6||fd|i|y#1swYTxYw#1swYHxYw)aL Saves a Keras model to save_directory in SavedModel format. Use this if you're using the Functional or Sequential APIs. Args: model (`Keras.Model`): The [Keras model](https://www.tensorflow.org/api_docs/python/tf/keras/Model) you'd like to save. The model must be compiled and built. save_directory (`str` or `Path`): Specify directory in which you want to save the Keras model. config (`dict`, *optional*): Configuration object to be saved alongside the model weights. include_optimizer(`bool`, *optional*, defaults to `False`): Whether or not to include optimizer in serialization. plot_model (`bool`, *optional*, defaults to `True`): Setting this to `True` will plot the model and put it in the model card. Requires graphviz and pydot to be installed. tags (Union[`str`,`list`], *optional*): List of tags that are related to model or string of a single tag. See example tags [here](https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1). model_save_kwargs(`dict`, *optional*): model_save_kwargs will be passed to [`tf.keras.models.save_model()`](https://www.tensorflow.org/api_docs/python/tf/keras/models/save_model). Nz>Called a Tensorflow-specific function but could not import it.z+Model should be built before trying to saveT)parentsexist_okzAProvided config to save_pretrained_keras should be a dict. Got: ''wrd task_namez>`task_name` input argument is deprecated. Pass `tags` instead.z history.jsonzZ`history.json` file already exists, it will be overwritten by the history of this version.zutf-8)encoding)indent sort_keysrc)r< ImportErrorbuilt ValueErrorrmkdirr,r2 RuntimeErrortyper CONFIG_NAMEopenjsondumpliststrpopwarningswarn FutureWarningr1rrY UserWarningramodels save_model) r"rQrbrcrPrdmodel_save_kwargsfrTrjr[s r%save_pretrained_kerasrsF }Z[[ ;;FGG.)N5&$'!bcghncobppqrs sy444 : :3 ?1 IIfa @H$ D#  6!%%k48I L   X  V  # #I . ){HV  }} == B &!N2D{{} p31Q %--//1M2unj(C LLE>lEVlZklA@ ?821sG<.H <H HKerasModelHubMixinc,tj|i|S)a Instantiate a pretrained Keras model from a pre-trained model from the Hub. The model is expected to be in `SavedModel` format. Args: pretrained_model_name_or_path (`str` or `os.PathLike`): Can be either: - A string, the `model id` of a pretrained model hosted inside a model repo on huggingface.co. Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a user or organization name, like `dbmdz/bert-base-german-cased`. - You can add `revision` by appending `@` at the end of model_id simply like this: `dbmdz/bert-base-german-cased@main` Revision is the specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. - A path to a `directory` containing model weights saved using [`~transformers.PreTrainedModel.save_pretrained`], e.g., `./my_model_directory/`. - `None` if you are both providing the configuration and state dictionary (resp. with keyword arguments `config` and `state_dict`). force_download (`bool`, *optional*, defaults to `False`): Whether to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist. proxies (`Dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request. token (`str` or `bool`, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated when running `transformers-cli login` (stored in `~/.huggingface`). cache_dir (`Union[str, os.PathLike]`, *optional*): Path to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used. local_files_only(`bool`, *optional*, defaults to `False`): Whether to only look at local files (i.e., do not try to download the model). model_kwargs (`Dict`, *optional*): model_kwargs will be passed to the model during initialization Passing `token=True` is required when you want to use a private model. )rfrom_pretrained)r#r$s r%from_pretrained_kerasrsj  - -t >v >>r'z'Push Keras model using huggingface_hub.)rbcommit_messageprivate api_endpointtokenbranch create_prallow_patternsignore_patternsdelete_patternslog_dirrcrdrPrepo_idrrrrrrrrrrc t|}|j|||dj}t5}t ||z }t ||f|| ||d|| 9| gnt | tr| gn| } | jdt| |dz |jd||||||| | | cdddS#1swYyxYw) a Upload model checkpoint to the Hub. Use `allow_patterns` and `ignore_patterns` to precisely filter which files should be pushed to the hub. Use `delete_patterns` to delete existing remote files in the same commit. See [`upload_folder`] reference for more details. Args: model (`Keras.Model`): The [Keras model](`https://www.tensorflow.org/api_docs/python/tf/keras/Model`) you'd like to push to the Hub. The model must be compiled and built. repo_id (`str`): ID of the repository to push to (example: `"username/my-model"`). commit_message (`str`, *optional*, defaults to "Add Keras model"): Message to commit while pushing. private (`bool`, *optional*): Whether the repository created should be private. If `None` (default), the repo will be public unless the organization's default is private. api_endpoint (`str`, *optional*): The API endpoint to use when pushing the model to the hub. token (`str`, *optional*): The token to use as HTTP bearer authorization for remote files. If not set, will use the token set when logging in with `huggingface-cli login` (stored in `~/.huggingface`). branch (`str`, *optional*): The git branch on which to push the model. This defaults to the default branch as specified in your repository, which defaults to `"main"`. create_pr (`boolean`, *optional*): Whether or not to create a Pull Request from `branch` with that commit. Defaults to `False`. config (`dict`, *optional*): Configuration object to be saved alongside the model weights. allow_patterns (`List[str]` or `str`, *optional*): If provided, only files matching at least one pattern are pushed. ignore_patterns (`List[str]` or `str`, *optional*): If provided, files matching any of the patterns are not pushed. delete_patterns (`List[str]` or `str`, *optional*): If provided, remote files matching any of the patterns will be deleted from the repo. log_dir (`str`, *optional*): TensorBoard logging directory to be pushed. The Hub automatically hosts and displays a TensorBoard instance if log files are included in the repository. include_optimizer (`bool`, *optional*, defaults to `False`): Whether or not to include optimizer during serialization. tags (Union[`list`, `str`], *optional*): List of tags that are related to model or string of a single tag. See example tags [here](https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1). plot_model (`bool`, *optional*, defaults to `True`): Setting this to `True` will plot the model and put it in the model card. Requires graphviz and pydot to be installed. model_save_kwargs(`dict`, *optional*): model_save_kwargs will be passed to [`tf.keras.models.save_model()`](https://www.tensorflow.org/api_docs/python/tf/keras/models/save_model). Returns: The url of the commit of your model in the given repository. )endpointT)rrrrg)rbrcrdrPNzlogs/*logsr") repo_typer folder_pathrrrevisionrrrr) r create_reporrrrr,rzr1r upload_folder)r"rrbrrrrrrrrrrrcrdrPrapitmp saved_paths r%push_to_hub_kerasr(s`  &CoogUGVZo[ccG !S#Y(    /!      #*"/37%%(   " "8 , Wj61 2  "))++!  5 " ! !s A6B66B?c@eZdZdZdZe ddeeee ffdZ y)raA Implementation of [`ModelHubMixin`] to provide model Hub upload/download capabilities to Keras models. ```python >>> import tensorflow as tf >>> from huggingface_hub import KerasModelHubMixin >>> class MyModel(tf.keras.Model, KerasModelHubMixin): ... def __init__(self, **kwargs): ... super().__init__() ... self.config = kwargs.pop("config", None) ... self.dummy_inputs = ... ... self.layer = ... ... def call(self, *args): ... return ... >>> # Initialize and compile the model as you normally would >>> model = MyModel() >>> model.compile(...) >>> # Build the graph by training it or passing dummy inputs >>> _ = model(model.dummy_inputs) >>> # Save model weights to local directory >>> model.save_pretrained("my-awesome-model") >>> # Push model weights to the Hub >>> model.push_to_hub("my-awesome-model") >>> # Download and initialize weights from the Hub >>> model = MyModel.from_pretrained("username/super-cool-model") ``` ct||yN)r)selfrQs r%_save_pretrainedz#KerasModelHubMixin._save_pretraineds dN3r'Nrbc  t tdtjj |st |||dt } n|} tjj| } | | _ | S)aHere we just call [`from_pretrained_keras`] function so both the mixin and functional APIs stay in sync. TODO - Some args above aren't used since we are calling snapshot_download instead of hf_hub_download. z>Called a TensorFlow-specific function but could not import it.r<)rr cache_dirrVlibrary_version) r<rorZr[isdirr rr load_modelrb) clsmodel_idrrforce_downloadproxiesresume_downloadlocal_files_onlyrrb model_kwargsstorage_folderr"s r%_from_pretrainedz#KerasModelHubMixin._from_pretrainedsn( =^_ _ww}}X&. !#$ . 0 N&N ''7  r'r) r! __module__ __qualname____doc__r classmethodr rrzrrr'r%rrs=!F4,0(c3h(((r'))TN)NFTN)rr):collections.abcabcr-rwrZr| functoolsrpathlibrshutilrtypingrrr r r huggingface_hubr r huggingface_hub.utilsrrrrrrrhf_apirrOrrr utils._typingr get_loggerr!loggerr<tf_kerasro tensorflowtfr(r0rBrRboolr2rarzryrrrrrr'r%rs% 33<HH$   H %     i :  "# )')')'tn )'X(,#'+ Pm#t)$Pm T#s(^ $Pm Pm  Pm 5s# $ PmPmf5?p "C""& $6:7;7;!#'+#w w  TN w  w d^ w 3-w  C=w  SMw ~w U49c>23w eDIsN34w eDIsN34w c]w w 5s# $!w "#w w tPPC sEE65E6