g@ ddlZddlZddlZddlmZmZmZmZddlm Z m Z m Z ddl m Z mZddlmZddlddlmZ d,de d eed eefd Z d,d eeede d eed eeeeffd Zded efdZdeded efdZdededed efdZdedededee d ef dZdededee ded ef dZde dedefdZdefdZde d efd Z de d efd!Z!d"ed efd#Z"d$e#fd%Z$d&e%d eefd'Z&d&e%d eefd(Z'ded efd)Z(ded efd*Z)d+Z*y)-N)AnyListOptionalTuple) HTTPExceptionRequeststatus)Router provider_list)verbose_proxy_logger)*)#CONFIGURABLE_CLIENTSIDE_AUTH_PARAMSrequestuse_x_forwarded_forreturncd}|durd|jvr|jd}|S|j|jj}|Sd}|S)NTzx-forwarded-for)headersclienthost)rr client_ips R/var/www/openai/venv/lib/python3.12/site-packages/litellm/proxy/auth/auth_utils.py_get_request_ip_addressrsdId"'8GOO'KOO$56   #NN''    allowed_ipsc:|yt||}||vrd|fSd|fS)z) Returns if ip is allowed or not )TN)rrFT)r)rrrrs r_check_valid_iprs>(-@I  #i ?r request_bodyc\d}|jd}|yd|vs d|vsd|vsd|vryd|vry y) zh if 'api_base' in request body. Check if complete credentials given. Prevent malicious attacks. NmodelF sagemakerbedrock vertex_aivertex_ai_betaapi_keyT)get)r given_models rcheck_complete_credentialsr(4sW"&K""7+K {"  # + % { *L  rrequest_body_value regex_strc<tj||s||k(ryy)zP Check if request_body_value matches the regex_str or is equal to param TF)rematchr)r*s rcheck_regex_or_str_matchr/Ms! xx -.)?Q2Q rparam#configurable_clientside_auth_paramsc|y|D]B}t|tr||k(ryt|ts+|dk(s1t||dsByy)zd Check if param is a str or dict and if request_body_value is in the list of allowed values FTapi_baser.) isinstancestrDictr/)r0r)r1items r_is_param_allowedr8VsY+23 dC Ud] d # "'?#5z*(4 rr llm_routerc|y|j|}|@|jdddtvr%|j|jddd}|y| |jyt |||jS)z Check if model is allowed to use configurable client-side params - get matching model - check if 'clientside_configurable_parameters' is set for model - F) model_group/r)r0r)r1)get_model_group_infosplitr r1r8)r r0r)r9 model_infos r5_allow_model_level_clientside_configurable_parametersrAns00U0CJ ;;sA q !] 2#88!KKQ/29JZKKS -,6,Z,Z rgeneral_settingscddg}|D]M}||vst|r|jdduryt|||||durytd|dy) u Check if the request body is safe. A malicious user can set the api_base to their own domain and invoke POST /chat/completions to intercept and steal the OpenAI API key. Relevant issue: https://huntr.com/bounties/4001e1a2-7b7a-4776-a3ae-e6692ec3d997 r3base_url)rallow_client_side_credentialsT)r r0r)r9zRejected Request: z is not allowed in request body. Enable with `general_settings::allow_client_side_credentials` on proxy config.yaml. Relevant Issue: https://huntr.com/bounties/4001e1a2-7b7a-4776-a3ae-e6692ec3d997)r(r&rA ValueError)rrBr9r banned_paramsr0s ris_request_body_saferHs ,M \ !.) ##$CDLE'3E':)   $UG,bb '2 r request_dataroutec &Kddlm}m}m}t |d{t ||||j ddt|j dd|j d d | \}}|sttjd |d d|vrx|d}|dur0tjdtjj||vr:tjd|d|ttjd|dyy7w)ax 1. Checks if request size is under max_request_size_mb (if set) 2. Check if request body is safe (example user has not set api_base in request body) 3. Check if IP address is allowed (if set) 4. Check if request route is an allowed route on the proxy (if set) Returns: - True Raises: - HTTPException if request fails initial auth checks r)rBr9 premium_user)rNr r)rrBr9r rrF)rrrzAccess forbidden: IP address z not allowed.) status_codedetailallowed_routesTz=Trying to set allowed_routes. This is an Enterprise feature. zRoute z not in allowed_routes=zAccess forbidden: Route z not allowed)litellm.proxy.proxy_serverrBr9rLcheck_if_request_size_is_saferHr&rrr HTTP_403_FORBIDDENr errorCommonProxyErrorsnot_premium_uservalue) rrIrJrBr9rL is_valid_ip passed_in_ip_allowed_routess rpre_db_read_auth_checksrZsB"VU ( 888!) R !0$((=,001FN!K 112<. N  ++*+;< t # & &OPaPrPrPxPxOyz   ' & &66GH  "551% E  ( ,59sDDC3D current_routecddlm}m} |dury|y|jdg}||vryy#t$r+}t j dt|Yd}~yd}~wwxYw)a Helper to check if the user defined public_routes on config.yaml Parameters: - current_route: str - the route the user is trying to call Returns: - bool - True if the route is defined in public_routes - bool - False if the route is not defined in public_routes In order to use this the litellm config.yaml should have the following in general_settings: ```yaml general_settings: master_key: sk-1234 public_routes: ["LiteLLMRoutes.public_routes", "/spend/calculate"] ``` rrBrLTFN public_routesz"route_in_additonal_public_routes: )rPrBrLr& Exceptionr rSr5)r[rBrLroutes_definedes r route_in_additonal_public_routesrbsn,J t #  #)--orB N * ""%GAx#PQs*** A!AAc t|drr|jjj|jjr9|jjt |jjdz dS|jjS#t $rX}tjdt|d|jj|jjcYd}~Sd}~wwxYw)z Helper to get the route from the request remove base url from path if set e.g. `/genai/chat/completions` -> `/chat/completions rDr=Nzerror on get_request_route: z!, defaulting to request.url.path=) hasattrurlpath startswithrDlenr_r debugr5)rras rget_request_routerjs  7J 'GKK,<,<,G,G    ! !- ;;##C(8(8(=(=$>$B$DE E;;## #  ""*3q6(2ST[T_T_TdTdSe f {{  s%A=BB C7A C2,C72C7cKddlm}m}|jdd}|/|dur1t j dt jjy|jjd}|rct|}t|}t jd |||kDr.td |d |d tjjd dy|j!d{}t#|}t|} t jd| | |kDr.td | d |d tjjd dy7gw)a Enterprise Only: - Checks if the request size is within the limit Args: request (Request): The incoming request. Returns: bool: True if the request size is within the limit Raises: ProxyException: If the request size is too large rr]max_request_size_mbNTzPusing max_request_size_mb - not checking - this is an enterprise only feature. content-length bytes_valuez"content_length request size in MB=z+Request size is too large. Request size is  MB. Max size is  MBmessagetypecoder0z request body request size in MB=)rPrBrLr&r warningrTrUrVrint bytes_to_mbriProxyExceptionProxyErrorTypesbad_request_errorbodyrh) rrBrLrlcontent_length header_sizeheader_size_mbr} body_sizerequest_size_mbs rrQrQ2sJ*../DdK& t # ( (bctdFdFdLdLcMN !,,-=> n-K([AN & &4^4DE  33$I.IYYjk~j@CD(::@@* . !'DD I)i@O & &2?2CD !44$I/IZZkllAADE(::@@*  (sC&E(E)A(EresponsecKddlm}m}|jdd}||dur1t j dt jjyttj|}t jd|||kDr.td |d |d tjjd d yw)a  Enterprise Only: - Checks if the response size is within the limit Args: response (Any): The response to check. Returns: bool: True if the response size is within the limit Raises: ProxyException: If the response size is too large rr]max_response_size_mbNTzQusing max_response_size_mb - not checking - this is an enterprise only feature. rnzresponse size in MB=z-Response size is too large. Response size is rprqrrrmrs)rPrBrLr&r rwrTrUrVrysys getsizeofrirzr{r|)rrBrLrresponse_size_mbs rcheck_response_size_is_saferqs J+//0FM' t # ( (cdueGeGeMeMdNO &3==3JK""%9:J9K#LM 2 2 GHXGYYjkkAADE$66<<&   sB=B?roc |dz S)z' Helper to convert bytes to MB irns rryrys + &&ruser_api_key_dictc|jrd|jvr|jdSy|jr9i}|jjD]\}}d|vs |d|d||<|Sy)Nmodel_rpm_limit rpm_limit)metadatamodel_max_budgetitems)rrr budgets rget_key_model_rpm_limitrs!!  1 : : :$--.?@ @   + +*,.??EEGME6f$ )<)H)/ )<&H rc|jrd|jvr|jdSy|jrd|jvr|jdSy)Nmodel_tpm_limit tpm_limit)rr)rs rget_key_model_tpm_limitrs_!!  1 : : :$--.?@ @   + + +<< <$55kB B rc$dg}|D]}||vsyy)Nz vertex-aiTFr)rJ%PROVIDER_SPECIFIC_PASS_THROUGH_ROUTESprefixs ris_pass_through_provider_routers(-) 8 U?8 rcHddlm}m}|dury|jddduryy)a Use this to decide if the rest of the LiteLLM Virtual Key auth checks should run on /vertex-ai/{endpoint} routes Use this to decide if the rest of the LiteLLM Virtual Key auth checks should run on provider pass through routes ex /vertex-ai/{endpoint} routes Run virtual key auth if the following is try: - User is premium_user - User has enabled litellm_setting.use_client_credentials_pass_through_routes rr]TF*use_client_credentials_pass_through_routes)rPrBrLr&)rJrBrLs r.should_run_auth_on_pass_through_provider_routers8J4 I5Q   rctjdd}tjdd}tjdd}|duxs |duxs|du}|S)z Check if the user has set up single sign-on (SSO) by verifying the presence of Microsoft client ID, Google client ID or generic client ID and UI username environment variables. Returns a boolean indicating whether SSO has been set up. MICROSOFT_CLIENT_IDNGOOGLE_CLIENT_IDGENERIC_CLIENT_ID)osgetenv)microsoft_client_idgoogle_client_idgeneric_client_id sso_setups r_has_user_setup_ssorsk ))$94@yy!3T: "5t< D ( + D ( + T ) r)F)+rr,rtypingrrrrfastapirrr litellmr r litellm._loggingr litellm.proxy._typeslitellm.types.routerrboolr5rrdictr(r/r8rArHrZrbrjrQrrxryUserAPIKeyAuthrrrrrrrrrs$ --22)1"D=B   +3D>  c] $+0$s)$ "$ 4#  .Td2 *M 0 03AI&AQ B$$*.$