g" UddlZddlmZddlZddlmZmZmZmZm Z ddl Z ddl m Z ddl mZddlddlmZddlmZdd lmZddleZeaeed <d efd Zd#d eefdZdefdZde de dee dee d ejBf dZ"ejGdgdddgdejGdgdddgde ded efd!Z$ded e fd"Z%y)$N)Optional) APIRouter HTTPExceptionRequestResponsestatus)verbose_proxy_logger) vertex_fine_tuning_apis_instance)*)user_api_key_auth)create_pass_through_route)get_secret_strdefault_vertex_configreturncTttdtdtdS)a Helper to get vertex pass through config from environment variables The following environment variables are used: - DEFAULT_VERTEXAI_PROJECT (project id) - DEFAULT_VERTEXAI_LOCATION (location) - DEFAULT_GOOGLE_APPLICATION_CREDENTIALS (path to credentials file) DEFAULT_VERTEXAI_PROJECTDEFAULT_VERTEXAI_LOCATION&DEFAULT_GOOGLE_APPLICATION_CREDENTIALS)vertex_projectvertex_locationvertex_credentials)VertexPassThroughCredentialsrg/var/www/openai/venv/lib/python3.12/site-packages/litellm/proxy/vertex_ai_endpoints/vertex_endpoints.py_get_vertex_env_varsrs, (%&@A&'BC)*RS rconfigc| tayt|trS|j D]@\}}t|t s|j ds)tj|||<Btdi|ay)aLSets vertex configuration from provided config and/or environment variables Args: config (Optional[dict]): Configuration dictionary Example: { "vertex_project": "my-project-123", "vertex_location": "us-central1", "vertex_credentials": "os.environ/GOOGLE_CREDS" } Nz os.environ/r) rr isinstancedictitemsstr startswithlitellm get_secretr)rkeyvalues rset_default_vertex_configr('sn~ 4 6&$ ,,.JC%%%*:*:=*I%007s )9B6Brec tjdjt|tjt j t|tr\tt|dt|jt|ddt|ddt|dtjSt|}tt|d|t|ddt|ddt|ddS) NzLlitellm.proxy.proxy_server.v1/projects/tuningJobs(): Exception occurred - {}messagetypeNoneparam status_code)r+r,r.codei)r errorformatr"debug traceback format_excrrProxyExceptiongetattrdetailrHTTP_400_BAD_REQUEST)r) error_msgs rexception_handlerr;AsV]] F  y3356!]#Ay#ahh-8FF+!Wf-M6+F+FG   1vh Ay)4FF+!Wf-M3/   rbase_urlrequested_routedefault_vertex_locationdefault_vertex_projectctj|}d|vr|j|}|S d}d|vrd}dj|||}d|z|z}|j|}|S)a< Allow user to specify their own project id / location. If missing, use defaults Handle cachedContent scenario - https://github.com/BerriAI/litellm/issues/5460 Constructed Url: POST https://LOCATION-aiplatform.googleapis.com/{version}/projects/PROJECT_ID/locations/LOCATION/cachedContents locations)pathv1 cachedContentv1beta1z{}/projects/{}/locations/{}/)httpxURL copy_withr2) r<r=r>r? new_base_url updated_urlvertex_versionbase_requested_routeupdated_requested_routes rconstruct_target_urlrOYs 99X&Lo%",,/,B  04N/)"8??.0G"$88?J((.E(FK rz/vertex-ai/{endpoint:path})GETPOSTPUTDELETEPATCHzVertex AI Pass-throughz pass-throughF)methodstagsinclude_in_schemaz/vertex_ai/{endpoint:path})rUrVendpointrequestfastapi_responsec ,Ktj|j}ddl}t j d|i}t |}t||d{}d}d} tjt|jxsi}t j d||jd|} | r| jdnd} d | d } |jd d|jd dntj}tj} tj } d | d } t#j$| |d d{\} }t#j&d| d| || dd d \}}dd|i}|}t j d||j)dsd|z}t+| || |}t j d|t-|}d}dt-|vrd}|dz }t/|||}|||||d{}|S777 w)zp Call LiteLLM proxy via Vertex AI SDK. [Docs](https://docs.litellm.ai/docs/pass_through/vertex_ai) rNzrequested endpoint %s)rY)rYapi_keyz;default_vertex_config not set, incoming request headers %sz/locations/([^/]+)zhttps://z-aiplatform.googleapis.com/zcontent-lengthhostvertex_ai_beta) credentials project_idcustom_llm_providerF) model auth_headergemini_api_keyrrrstreamrbapi_base AuthorizationBearer zrequest_route %srF)r<r=r>r?zupdated url %srgTz?alt=sse)rXtargetcustom_headers)rg)rGrHrBrer r3get_litellm_virtual_keyr rrr headerssearchgrouppoprrr _ensure_access_token_async_get_token_and_urlr#rOr"r )rXrYrZencoded_endpointrmroapi_key_to_useuser_api_key_dictrrmatchbase_target_urlr _auth_headerre_ request_routerKrkis_streaming_request endpoint_funcreceived_values rvertex_proxy_routers_*yy*// 6AG,W=N/ NO++3w'-2"" I7  /:,1%++a.t$_$55PQ $d+ FD!.==/??2EE$_$55PQ3MM.)$4  % n:LL$1)+ 0   Q w{m4 %M1=A  & &s +!11' ( /- K/= F 3{###*.M )# N u8 ns8AHH C*HHCHHHHHc~|jjd}|rd|S|jjddS)z Extract and format API key from request headers. Prioritizes x-litellm-api-key over Authorization header. Vertex JS SDK uses `Authorization` header, we use `x-litellm-api-key` to pass litellm virtual key zx-litellm-api-keyrjrirc)roget)rYlitellm_api_keys rrnrnsAoo))*=>O)** ??   33r)N)&r4typingrrGfastapirrrrrr$litellm._loggingr litellm.fine_tuning.mainr litellm.proxy._types$litellm.proxy.auth.user_api_key_authr ;litellm.proxy.pass_through_endpoints.pass_through_endpointsr litellm.secret_managers.mainr-litellm.types.passthrough_endpoints.vertex_airouterrr__annotations__rr r( Exceptionr;r"rHrO api_routerrnrrrrsA GG1E"B8; 6R6T3T :  ChtnC4  0$$$&c]$%SM $  YY $N 5 "N 3   5 "N 3 kk kk  k\ 4W 4 4r