gA UdZddlZddlZddlZddlZddlmZddlmZddl m Z ddl m Z m Z mZmZmZmZmZmZmZmZmZmZmZmZddlmZddlmZmZmZm Z m!Z!m"Z"d d l#m$Z$d d l%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+d d l,m-Z-m.Z.e rddl/m0Z0m1Z1ddl2m3Z3e4Z5ee4e fZ6ee7efZ8ee8e6e5fZ9ddhZ:ejve<Z=eGddZ>dZ?dZ@dZAdaBeee4ee4feCd<dee4ee4ffdZDdee dee fdZEede9dee8fdZFededdeedfdZFedee9deee8ddffdZFde9de4fd ZGd!e4dd"fd#ZHde7defd$ZIde7defd%ZJde7dd"fd&ZK dDd'ee4ee4e fe9fd(eee4e fd)eLdee4e ffd*ZMd+ee7d,eLdeee4ee.ffd-ZNd+ee7d,eLdeee4ee.ffd.ZOd/e7d,eLdeee4e.ffd0ZPd1ee7dee-fd2ZQd1ee7dee-fd3ZRd/e7dee-fd4ZSd5d6d7d8dee7fd9ZTiZUeee4ee4feCd:<d;ee4dZWd?edefd@ZXdAee4dBee4de fdCZYy)EzEContains utilities used by both the sync and async inference clients.N)contextmanager) dataclass)Path) TYPE_CHECKINGAny AsyncIterableBinaryIOContextManagerDict GeneratorIterableListLiteralNoReturnOptionalUnionoverload) HTTPError)GenerationErrorIncompleteGenerationErrorOverloadedErrorTextGenerationError UnknownErrorValidationError)ENDPOINT)build_hf_headers get_sessionhf_raise_for_statusis_aiohttp_availableis_numpy_availableis_pillow_available)ChatCompletionStreamOutputTextGenerationStreamOutput)ClientResponse ClientSessionImagez text-to-imagezimage-to-imagec:eZdZUdZeed<eed<eed<eed<y) ModelStatusa This Dataclass represents the model status in the Hugging Face Inference API. Args: loaded (`bool`): If the model is currently loaded into Hugging Face's InferenceAPI. Models are loaded on-demand, leading to the user's first request taking longer. If a model is loaded, you can be assured that it is in a healthy state. state (`str`): The current state of the model. This can be 'Loaded', 'Loadable', 'TooBig'. If a model's state is 'Loadable', it's not too big and has a supported backend. Loadable models are automatically loaded when the user first requests inference on the endpoint. This means it is transparent for the user to load a model, except that the first call takes longer to complete. compute_type (`Dict`): Information about the compute resource the model is using or will use, such as 'gpu' type and number of replicas. framework (`str`): The name of the framework that the model was built with, such as 'transformers' or 'text-generation-inference'. loadedstate compute_type frameworkN)__name__ __module__ __qualname____doc__bool__annotations__strr V/var/www/openai/venv/lib/python3.12/site-packages/huggingface_hub/inference/_common.pyr+r+Qs, L JNr8r+c8ts tdddl}|S)NzMPlease install aiohttp to use `AsyncInferenceClient` (`pip install aiohttp`).r)r ImportErroraiohttp)r<s r9_import_aiohttpr=rs  !ijj Nr8c8ts tdddl}|S)z.Make sure `numpy` is installed on the machine.zGPlease install numpy to use deal with embeddings (`pip install numpy`).rN)r!r;numpy)r?s r9 _import_numpyr@{s  cdd Lr8c<ts tdddlm}|S)z,Make sure `PIL` is installed on the machine.zPlease install Pillow to use deal with images (`pip install Pillow`). If you don't want the image to be post-processed, use `client.post(...)` and get the raw response from the server.rr()r"r;PILr)r(s r9_import_pil_imagerCs%   `   Lr8_RECOMMENDED_MODELSreturnc  tstjtdt }t ||j jDcic]\}}|t|dc}}atScc}}w)Nz /api/tasks)headers widgetModels) rDrgetrrrjsonitems_first_or_none)responsetaskdetailss r9_fetch_recommended_modelsrPs|"=$$z%tj d|t j |jyt|}|jstd|dt|tr7tj d||jd5}|dddy|y#1swYyxYww) zOpen `content` as a binary file, either from a URL, a local path, or raw bytes. Do nothing if `content` is None, TODO: handle a PIL.Image as input TODO: handle base64 as input zhttps://zhttp://zDownloading content from NzFile not found at z. If `data` is a string, it must either be a URL or a path to a local file. To pass raw content, please encode it as bytes first.zOpening content from rb) isinstancer6 startswithloggerdebugrrIrSrexistsFileNotFoundErroropen)rSfs r9rWrWs'3   j )W-?-? -J LL4WI> ?-##G,44 4 w-~~#$WI.OO  '4  ,WI67 \\$ 1G   sCC5C)C5)C2.C5ct|5}t|tr|n|j}t j |j cdddS#1swYyxYw)z\Encode a raw file (image, audio) into base64. Can be bytes, an opened file, a path or a URL.N)rWr[bytesreadbase64 b64encodedecode)rSdata data_as_bytess r9 _b64_encoderksE  !T *4 7TYY[  .557 " ! !s AAA$ encoded_imager)ct}|jtjt j |S)z/Parse a base64-encoded string into a PIL Image.)rCraioBytesIOrf b64decode)rlr)s r9 _b64_to_imagerqs/  E 5::bjj!1!1-!@A BBr8cHtj|jS)acParse bytes from a Response object into a Python list. Expects the response body to be JSON-encoded data. NOTE: This is exactly the same implementation as `_bytes_to_dict` and will not complain if the returned data is a dictionary. The only advantage of having both is to help the user (and mypy) understand what kind of data to expect. rJloadsrhrVs r9_bytes_to_listru ::gnn& ''r8cHtj|jS)acParse bytes from a Response object into a Python dictionary. Expects the response body to be JSON-encoded data. NOTE: This is exactly the same implementation as `_bytes_to_list` and will not complain if the returned data is a list. The only advantage of having both is to help the user (and mypy) understand what kind of data to expect. rsrVs r9_bytes_to_dictrxrvr8c`t}|jtj|S)zParse bytes from a Response object into a PIL Image. Expects the response body to be raw bytes. To deal with b64 encoded images, use `_b64_to_image` instead. )rCrarnro)rSr)s r9_bytes_to_imagerzs&  E 5::bjj) **r8inputs parameters expect_binarych|i}|jDcic] \}}| || }}}t|dkD}t|ttf}|r |st|t st d||r|sd|iS|s|rt d|i}|rt||d<n||d<|r||d<d|iScc}}w)a$ Used in `InferenceClient` and `AsyncInferenceClient` to prepare the payload for an API request, handling various input types and parameters. `expect_binary` is set to `True` when the inputs are a binary object or a local path or URL. This is the case for image and audio inputs. rz5Expected binary inputs or a local path or a URL. Got rizUnexpected binary inputs. Got r{r|rJ)rKlenr[rdrr6 ValueErrorrk)r{r|r}kvhas_parameters is_binaryrJs r9_prepare_payloadrs #-#3#3#5G#541a!Q$#5JG_q(N6E4=1IYz&#/FPQWPXYZZ^ Y9&BCCD$V,X X'\ D>1Hs B.B.bytes_output_as_linesrOc#bK|D]} t||}||y#t$rYywxYww)z*Used in `InferenceClient.text_generation`.N%_format_text_generation_stream_output StopIterationrrO byte_payloadoutputs r9 _stream_text_generation_responser/sC .  :<QF  L .   s / // ,/,/ctK|23d{} t||}||7#t$rYywxYw6yw)z/Used in `AsyncInferenceClient.text_generation`.Nrrs r9&_async_stream_text_generation_responser=sN 4l :<QF  L    4s486$68 &886 3838rc|jdsy|jdk(r td|jd}t j |j djd}|jdt|d|jdtj|}|s|jjS|S Nsdata:s data: [DONE]z[DONE] signal received.zutf-8zdata:z/nerror error_type)r\striprrhrJrtlstriprstriprI_parse_text_generation_errorr%parse_obj_as_instancetokentext)rrOpayload json_payloadrs r9rrKs  " "8 ,.566!!'*G::gnnW5<EcI\I\IaIaIc G$[[. 0 C Z'  s A A A,rrc|dk(r t|S|dk(r t|S|dk(r t|S|dk(r t|St |S)N generationincomplete_generation overloaded validation)rrrrr)rrs r9rrsZ\!u%%,,(//\!u%%\!u%%  r8)F)Zr3rfrnrJlogging contextlibr dataclassesrpathlibrtypingrrrr r r r r rrrrrrrequestsrhuggingface_hub.errorsrrrrrr constantsrutilsrrrr r!r"_generated.typesr$r%r<r&r' PIL.Imager)r6UrlTPathTrdBinaryTContentTTASKS_EXPECTING_IMAGES getLoggerr0r]r+r=r@rCrDr5rPrLrWrkrqrurxrzr4rrrrrrrrrrrrrr7r8r9rs7L %!"!U5  c4i x  %% &*+;<   8 $  @ ;?Xd3 #567>4Xc](:#;$s)  " "G" "  ( T](GDM"( ( Xh/Ihw>OQUW[>[4\>88c8CCC (E(d((E(d(+U+w+ # #tCH~x/ 0#c3h(## #s(^ #R #E? 59  8C=(#=> >?  (/ :>  = }-GH HI 88"&8 eC33458, %  ()  u% -. JJ ()J*O?OTabgTh0GI#T(3-c*B%CHY8C=YVZ[^V_YdhY>8C=>T#Y>I(6  8C= Uh r8