g"|dZddlZddlZddlmZddlmZmZmZmZddl m Z ddl m Z GddZ Gd d Zy) zA Class to handle llm wildcard routing and regex pattern matching N)Match)DictListOptionalTuple)get_llm_provider)verbose_router_loggerc peZdZededeeeffdZedeee efde eee effdZ y) PatternUtilspatternreturncNgd}ttfd|Df}|S)z Calculate pattern specificity based on length and complexity. Args: pattern: Regex pattern to analyze Returns: Tuple of (length, complexity) for sorting ) *+?\^$|()c3@K|]}j|ywN)count).0charr s c/var/www/openai/venv/lib/python3.12/site-packages/litellm/router_utils/pattern_match_deployments.py z=PatternUtils.calculate_pattern_specificity..s0@ d#0@s)lensum)r complexity_charsret_vals` rcalculate_pattern_specificityz*PatternUtils.calculate_pattern_specificitys6J L 0@   patternsc<t|jddS)z Cached property for patterns sorted by specificity. Returns: Sorted list of pattern-deployment tuples c2tj|dS)Nr)r r#)xs rz.PatternUtils.sorted_patterns../s,DDQqTJr$T)keyreverse)sorteditemsr%s rsorted_patternszPatternUtils.sorted_patterns#s! NN J  r$N) __name__ __module__ __qualname__ staticmethodstrrintr#rrr/r$rr r smsuS#X&  sDJ'  eCdO$ %    r$r c eZdZdZdZdedefdZdedefdZde d e ede efd Z dd e ed e e ede e efdZ ede dedefdZ ddede ede e efdZ ddede ede efdZy )PatternMatchRouterz Class to handle llm wildcard routing and regex pattern matching doc: https://docs.litellm.ai/docs/proxy/configs#provider-specific-wildcard-routing This class will store a mapping for regex pattern: List[Deployments] ci|_yrr.)selfs r__init__zPatternMatchRouter.__init__=s )+ r$r llm_deploymentc|j|}||jvrg|j|<|j|j|y)z Add a regex pattern and the corresponding llm deployments to the patterns Args: pattern: str llm_deployment: str or List[str] N)_pattern_to_regexr%append)r:r r<regexs r add_patternzPatternMatchRouter.add_pattern@sE&&w/  %#%DMM%  e##N3r$r cLtj|jddS)a0 Convert a wildcard pattern to a regex pattern example: pattern: openai/* regex: openai/.* pattern: openai/fo::*::static::* regex: openai/fo::.*::static::.* Args: pattern: str Returns: str: regex pattern z\*z(.*))reescapereplace)r:r s rr>z$PatternMatchRouter._pattern_to_regexNs ,yy!))%88r$matched_pattern deploymentscg}|D]K}tj|}tj||dd|dd<|j |M|S)Nlitellm_paramsmodel)rF litellm_deployment_litellm_model)copydeepcopyr8set_deployment_model_namer?)r:rFrGnew_deployments deploymentnew_deployments r#_return_pattern_matched_deploymentsz6PatternMatchRouter._return_pattern_matched_deploymentsfsp%J!]]:6N"<<$35?@P5Q6= + ,W 5  " "> 2&r$Nrequestfiltered_model_namesc |ytj|j}||Dcgc]}|j|c}ng}|D]9\}}|||vr t j ||}|s&|j ||cS ycc}w#t$r+} tjdt| Yd} ~ yd} ~ wwxYw)a Route a requested model to the corresponding llm deployments based on the regex pattern loop through all the patterns and find the matching pattern if a pattern is found, return the corresponding llm deployments if no pattern is found, return None Args: request: str - the received model name from the user (can be a wildcard route). If none, No deployments will be returned. filtered_model_names: Optional[List[str]] - if provided, only return deployments that match the filtered_model_names Returns: Optional[List[Deployment]]: llm deployments N)rFrGz#Error in PatternMatchRouter.route: ) r r/r%r>rCmatchrR Exceptionr debugr4) r:rSrTr/mregex_filtered_model_namesr llm_deployments pattern_matches rroutezPatternMatchRouter.routexs  X*::4==IO(35II4Hq''*4HI ' -<((4'AA "' :  CC(5?D-<%J X ! ' '*McRSfX(V W W Xs9B %B B,B /B B B C!B;;CrKcd|vr|S|jd}|j}t||kDr |jS|D]}|j d|d}|S)a Set the model name for the matched pattern llm deployment E.g.: Case 1: model_name: llmengine/* (can be any regex pattern or wildcard pattern) litellm_params: model: openai/* if model_name = "llmengine/foo" -> model = "openai/foo" Case 2: model_name: llmengine/fo::*::static::* litellm_params: model: openai/fo::*::static::* if model_name = "llmengine/foo::bar::static::baz" -> model = "openai/foo::bar::static::baz" Case 3: model_name: *meta.llama3* litellm_params: model: bedrock/meta.llama3* if model_name = "hello-world-meta.llama3-70b" -> model = "bedrock/meta.llama3-70b" r)rgroupsrstringrE)rFrKwildcard_countdynamic_segmentssegments rrNz,PatternMatchRouter.set_deployment_model_names}B 6 63 39??D+113  > 1&& (G/O/W/WWa0 ,( 0/r$rJcustom_llm_providerc| t|\}}}}|j|xs|j|d|S#t$rY4wxYw)z Check if a pattern exists for the given model and custom llm provider Args: model: str custom_llm_provider: Optional[str] Returns: bool: True if pattern exists, False otherwise )rJ/)rrWr^)r:rJrf_s r get_patternzPatternMatchRouter.get_patternsj  &  %51 ' zz% PDJJ2E1Faw/O$PP  s> A  A c2|j||}|r|SgS)z Get the deployments by pattern Args: model: str custom_llm_provider: Optional[str] Returns: List[Dict]: llm deployments matching the pattern )rj)r:rJrfr\s rget_deployments_by_patternz-PatternMatchRouter.get_deployments_by_patterns%((0CD   r$r)r0r1r2__doc__r;r4rrAr>rrrRrr^r3rNrjrlr6r$rr8r84s", 43 4 49990$37: d&SW(}(rqs8 ..$2# # LMMr$