gR dZddlZddlmZmZddlZddlmZmZmZm Z m Z ddl Z ddl m Z ddlddlmZeZej%dd geegd ej%d d geeg defdZej%dd geegd ej%dd geeg defdZdedeefdZej%dd gd eegej%dd geeg eefdedefdZej7dd geegeej7dd gd eegej:dfdefd Zej%d!d geeg ej%d"d gd eegeefde defd#Z!ej%d$d geeg ej%d%d gd eegeefde"defd&Z#ej7d'd geegeeej7d(d gd eegeefd)e defd*Z$y)+z} CUSTOMER MANAGEMENT All /customer management endpoints /customer/new /customer/info /customer/update /customer/delete N)ListOptional) APIRouterDepends HTTPExceptionRequeststatus)verbose_proxy_logger)*)user_api_key_authz/end_user/blockzCustomer ManagementF)tags dependenciesinclude_in_schemaz/customer/block)r rdatacKddlm} g}|\|jD]L}|jjj d|i|ddddid d{}|j |Nntd d d i d|iS7+#t$r>}tjdt|td d t|i d}~wwxYww)a [BETA] Reject calls with this end-user id Parameters: - user_ids (List[str], required): The unique `user_id`s for the users to block (any /chat/completion call with this user={end-user-id} param, will be rejected.) ``` curl -X POST "http://0.0.0.0:8000/user/block" -H "Authorization: Bearer sk-1234" -d '{ "user_ids": [, ...] }' ``` r prisma_clientNuser_idT)rblockedr)createupdatewherererrorzPostgres DB Not connected status_codedetail blocked_userszAn error occurred - ) litellm.proxy.proxy_serverruser_idsdblitellm_endusertableupsertappendr Exceptionr rstr)rrrecordsidrecordes j/var/www/openai/venv/lib/python3.12/site-packages/litellm/proxy/management_endpoints/customer_endpoints.py block_userr-s89G  $mm,//DDKK$b/.0T"B#,d"3 L v&$ !<=   )) G""%9#a&#BCWc!f4EFFGs;C AA?A=*A?<C =A?? C9CCC z/end_user/unblockz/customer/unblockczKddlmtfdtjDrtj t dddittj tr1|jD]!}tj j|#nt ddd id tj iSw) z [BETA] Unblock calls with this user id Example ``` curl -X POST "http://0.0.0.0:8000/user/unblock" -H "Authorization: Bearer sk-1234" -d '{ "user_ids": [, ...] }' ``` r)_ENTERPRISE_BlockedUserListc36K|]}t|yw)N) isinstance).0xr/s r, zunblock_user..nsVDUq 19:DUsrz:Blocked user check was never set. This call has no effect.rrzF`blocked_user_list` must be set as a list. Filepaths can't be updated.r) -enterprise.enterprise_hooks.blocked_user_listr/anylitellm callbacksblocked_user_listrr1listr!remove)rr)r/s @r, unblock_userr=Qs0 VGDUDUV V  $ $ ,U  '++T2--B  % % , ,R 0 a   W66 77sB8B;returnctjj}i}|D]}|dk(r t||d}||||<|r tdi|Sy)zE Return a new budget object if new budget params are passed. budget_idN)BudgetNewRequest model_fieldskeysgetattr)r budget_paramsbudget_kv_pairs field_namevalues r,new_budget_requestrJsh%11668MO$  $ j$/  */OJ ' $2/22 z /end_user/new)r rrz /customer/newuser_api_key_dictc @K ddlm}m}m}|'t ddt j ji |j|'t ddt jji|j|jvr@t dddj|jt|jii}t|}|r |jjj!i|j#d |j$xs||j$xs|d  d{}|j*|d <n|j*|j*|d <|j-d } | j/D]+\} } | t0j2j5vs'| || <-|jj6j!|dd id{} | S7#t&$r}t ddt)|id}~wwxYw71#t&$r}t9j:djt)|dt)|vrt=d|j$ddddt?|trKt=tA|ddt)|ddtA|ddtA|dtBjD t?|t<r|t=d!t)|zdtA|ddtBjD d}~wwxYww)"a Allow creating a new Customer Parameters: - user_id: str - The unique identifier for the user. - alias: Optional[str] - A human-friendly alias for the user. - blocked: bool - Flag to allow or disallow requests for this end-user. Default is False. - max_budget: Optional[float] - The maximum budget allocated to the user. Either 'max_budget' or 'budget_id' should be provided, not both. - budget_id: Optional[str] - The identifier for an existing budget allocated to the user. Either 'max_budget' or 'budget_id' should be provided, not both. - allowed_model_region: Optional[Union[Literal["eu"], Literal["us"]]] - Require all user requests to use models in this specific region. - default_model: Optional[str] - If no equivalent model in the allowed region, default all requests to this model. - metadata: Optional[dict] = Metadata for customer, store information for customer. Example metadata = {"data_training_opt_out": True} - budget_duration: Optional[str] - Budget is reset at the end of specified duration. If not set, budget is never reset. You can set duration as seconds ("30s"), minutes ("30m"), hours ("30h"), days ("30d"). - tpm_limit: Optional[int] - [Not Implemented Yet] Specify tpm limit for a given customer (Tokens per minute) - rpm_limit: Optional[int] - [Not Implemented Yet] Specify rpm limit for a given customer (Requests per minute) - model_max_budget: Optional[dict] - [Not Implemented Yet] Specify max budget for a given model. Example: {"openai/gpt-4o-mini": {"max_budget": 100.0, "budget_duration": "1d"}} - max_parallel_requests: Optional[int] - [Not Implemented Yet] Specify max parallel requests for a given customer. - soft_budget: Optional[float] - [Not Implemented Yet] Get alerts when customer crosses given budget, doesn't block requests. - Allow specifying allowed regions - Allow specifying default model Example curl: ``` curl --location 'http://0.0.0.0:4000/customer/new' --header 'Authorization: Bearer sk-1234' --header 'Content-Type: application/json' --data '{ "user_id" : "ishaan-jaff-3", "allowed_region": "eu", "budget_id": "free_tier", "default_model": "azure/gpt-3.5-turbo-eu" <- all calls from this user, use this model? }' # return end-user object ``` NOTE: This used to be called `/end_user/new`, we will still be maintaining compatibility for /end_user/XXX for these endpoints r)litellm_proxy_admin_name llm_routerrNrrrizmDefault Model not on proxy. Configure via `/model/new` or config.yaml. Default_model={}, proxy_model_names={}T) exclude_unset) created_by updated_by)rr@ exclude_nonelitellm_budget_table)rincludez\litellm.proxy.management_endpoints.customer_endpoints.new_end_user(): Exception occured - {}z3Unique constraint failed on the fields: (`user_id`)z(Customer already exists, passed user_id=z. Please pass a new user_id. bad_requestr5r)messagetypecodeparamrInternal Server Error()internal_errorr[NonerrXrYr[rZInternal Server Error, )#r rNrOrrCommonProxyErrorsdb_not_connected_errorrI default_model no_llm_routerget_model_namesformatsetrJr"litellm_budgettabler model_dumprr&r'r@dictitemsrBrCrDr#r exceptionProxyExceptionr1rEr HTTP_500_INTERNAL_SERVER_ERROR) rrLrNrOrnew_end_user_obj _new_budget budget_recordr+ _user_datakvend_user_records r, new_end_userrwsPp  .EEKKL  Q     )!# ##%6%D%D%J%JK##:+E+E+GG# #"Q"X"X ..J4N4N4P0Q""$).  " O&3&6&6&J&J&Q&Q%00t0D&7&?&?&[C[&7&?&?'43 'R'! -:,C,C [ ) ^^ ',0NN [ )YYDY1 $$&DAq(55::<<&' #' !. 0 0 E E L L!+T2!M!   7! O#Wc!f *G-A6!!Wf-66   / st5LB HAG*5G(6G*:A3H.3H!H"H'L(G** H3H  HH LC8LLLz/customer/info)r rresponse_modelz/end_user/infoz%End User ID in the request parameters) description end_user_idc6Kddlm}|'tddtjj i|j jjd|idd i d{}|td dd j|i|jd S76w)ae Get information about an end-user. An `end_user` is a customer (external user) of the proxy. Parameters: - end_user_id (str, required): The unique identifier for the end-user Example curl: ``` curl -X GET 'http://localhost:4000/customer/info?end_user_id=test-litellm-user-4' -H 'Authorization: Bearer sk-1234' ``` rrNrrrrrUT)rrVr5z#End User Id={} does not exist in dbrS) r rrrbrcrIr"r# find_firstrgrj)rzr user_infos r, end_user_infor~<s:9.EEKKL  $&&;;FF+&1G0NGIBII+VW     T  22sAB B!7Bz/customer/updatez/end_user/updatec Kddlm} |j}| tdi}|j D]\}}| |gidfvs|||<t j d||jt|jdkDr|j|d<t j d|jjjd|ji|d{}|td |jt j d ||Std |j7P#t$r}t jd jt|t j t!j"t%|t&rKt)t+|d dt|ddt+|ddt+|dt,j.t%|t(r|t)dt|zdt+|ddt,j.d}~wwxYww)a Example curl Parameters: - user_id: str - alias: Optional[str] = None # human-friendly alias - blocked: bool = False # allow/disallow requests for this end-user - max_budget: Optional[float] = None - budget_id: Optional[str] = None # give either a budget_id or max_budget - allowed_model_region: Optional[AllowedModelRegion] = ( None # require all user requests to use models in this specific region ) - default_model: Optional[str] = ( None # if no equivalent model in allowed region - default all requests to this model ) Example curl: ``` curl --location 'http://0.0.0.0:4000/customer/update' --header 'Authorization: Bearer sk-1234' --header 'Content-Type: application/json' --data '{ "user_id": "test-litellm-user-4", "budget_id": "paid_tier" }' See below for all params ``` rrNNot connected to DB!z$/customer/update: Received data = %srz,In update customer, user_id condition block.rzEFailed updating customer data. User ID does not exist passed user_id=8received response from updating prisma client. response=&user_id is required, passed user_id = zDlitellm.proxy.proxy_server.update_end_user(): Exception occured - {}rr\r]r^r[r_rr`ra)r rjsonr&rlr debugrlenr"r#r ValueErrorrrgr' traceback format_excr1rrnrEr ro) rrLr data_jsonnon_default_valuesrtruresponser+s r,update_end_userrms8Z99 ))+  23 3 OO%DAq}+" )*"1% & ""#I4P << #DLL(9A(=,0LL y ) & &'U V*--BBII $,,/6HJH [\`\h\h[ij ! & &J8*U OEdll^TU U  "" R Y YA  ""9#7#7#9: a ' 8/Ec!fXQ-OP%a&1Q v/T/TU   > *G-A6!!Wf-66   ! sHH>7D8D8 BD8%D6&7D8H>D88 H;C5H66H;;H>z/customer/deletez/end_user/deletec Kddlm} | tdtjd||j t |j trt|j dkDr|jjjdd|j iid{}|td |j |t|j k7r2td |j d |d t|j d tjd||dt|j zdStd|j 7#t$r}tjdjt|tjt!j"t |t$rKt't)|ddt|ddt)|ddt)|dt*j,t |t&r|t'dt|zdt)|ddt*j,d}~wwxYww)a Delete multiple end-users. Parameters: - user_ids (List[str], required): The unique `user_id`s for the users to delete Example curl: ``` curl --location 'http://0.0.0.0:4000/customer/delete' --header 'Authorization: Bearer sk-1234' --header 'Content-Type: application/json' --data '{ "user_ids" :["ishaan-jaff-5"] }' See below for all params ``` rrNrz$/customer/delete: Received data = %srin)rzEFailed deleting customer data. User ID does not exist passed user_id=zIFailed deleting all customer data. User ID does not exist passed user_id=z . Deleted z customers, passed z customersrz)Successfully deleted customers with ids: )deleted_customersrXrzDlitellm.proxy.proxy_server.delete_end_user(): Exception occured - {}rr\r]r^r[r_rr`ra)r rr&r rr!r1r;rr"r# delete_manyrr'rrgrrrrnrEr ro)rrLrrr+s r,delete_end_userrsvB96  23 3""#I4P MM %4==$/DMM"Q&*--BBNN 4"78OH [\`\i\i[jk3t}}-- _`d`m`m_nnxzByCCVWZ[_[h[hWiVjjtu ! & &J8*U &.Fdmm$%  Edmm_UV V).  "" R Y YA  ""9#7#7#9: a ' 8/Ec!fXQ-OP%a&1Q v/T/TU   > *G-A6!!Wf-66   ! s=IBE#E$BE>I?E I"C5IIIz/customer/listz/end_user/list http_requestc Kddlm}|jtjk7rE|jtj k7r(t dddj|ji|'t ddtjji|jjjd d i d{}g}|D]*}|jtd i|j!,|S77w) z [Admin-only] List all available customers Example curl: ``` curl --location --request GET 'http://0.0.0.0:4000/customer/list' --header 'Authorization: Bearer sk-1234' ``` rrirz&Admin-only endpoint. Your user role={}rNr5rUT)rVrA)r r user_roleLitellmUserRoles PROXY_ADMINPROXY_ADMIN_VIEW_ONLYrrgrbrcrIr"r# find_manyr%LiteLLM_EndUserTablerj)rrLrrreturned_responseitems r, list_end_userr6s49 ##'7'C'CC  ' '+;+Q+Q QAHH%//  .EEKKL  #%%::DD'.EH57  !5!J8I!JK sB=C9?C78C9)%__doc__rtypingrrfastapirrrrr r8litellm._loggingr litellm.proxy._types$litellm.proxy.auth.user_api_key_authr routerpost BlockUsersr-r=NewCustomerRequestrBrJUserAPIKeyAuthrwgetrQueryr'r~UpdateCustomerRequestrDeleteCustomerRequestrrrArKr,rs !FF1"B   +,-     +,-  (G:(G  (GV  +,-     +,-  '8Z'8  '8T/H=M4N(  +,-     +,- )00A(BT T %T  T n  +,-'     +,-  %w}};"3"3  "3J  +,-    +,-  )00A(B^ ^ %^  ^ B  +,-    +,-  )00A(BO O %O  O d  +,-,-     +,-  )00A(B**%*  *rK