g@dZddlZddlZddlZddlZddlZddlmZddlmZm Z m Z m Z ddl m Z ddlZddlmZddlmZddlmZmZmZdd ldd lmZdd lmZd ZGd deZy)ay DataDog Integration - sends logs to /api/v2/log DD Reference API: https://docs.datadoghq.com/api/latest/logs `async_log_success_event` - used by litellm proxy to send logs to datadog `log_success_event` - sync version of logging to DataDog, only used on litellm Python SDK, if user opts in to using sync functions async_log_success_event: will store batch of DD_MAX_BATCH_SIZE in memory and flush to Datadog once it reaches DD_MAX_BATCH_SIZE or every 5 seconds async_service_failure_hook: Logs failures from Redis, Postgres (Adjacent systems), as 'WARNING' on DataDog For batching specific details see CustomBatchLogger class N)datetime)AnyListOptionalUnion)Response)verbose_logger)CustomBatchLogger)_get_httpx_clientget_async_httpx_clienthttpxSpecialProvider)*)ServiceLoggerPayload)StandardLoggingPayloadceZdZfdZdZdZdZdZdZde e e fde d e jd e jd e f d Zd ed efdZ ddedeedee d ee eefd ee eefdee f dZ ddedeedee d ee eefd ee eefdee f dZde e e fde d e jd e jd e f dZedZedZedZedZedZedZ xZ!S) DataDogLoggerc D tjdtjdd t dtjdd t dt t j|_tjd|_ dtjdd |_ tjd xs,tjd xstjd }| |d |_ t|_ tj|jtj |_t%|Ldi||j"t(d y#t$r(}tj*dt-||d}~wwxYw)z Initializes the datadog logger, checks if the correct env variables are set Required environment variables: `DD_API_KEY` - your datadog api key `DD_SITE` - your datadog site, example = `"us5.datadoghq.com"` zDatadog: in init datadog logger DD_API_KEYNz)DD_API_KEY is not set, set 'DD_API_KEY=<>DD_SITEz+DD_SITE is not set in .env, set 'DD_SITE=<>) llm_providerzhttps://http-intake.logs.z /api/v2/logs_DATADOG_BASE_URLDATADOG_BASE_URL DD_BASE_URL) flush_lock batch_sizez.Datadog: Got exception on init Datadog client )r debugosgetenv Exceptionr r LoggingCallback async_clientr intake_urlr sync_clientasyncio create_taskperiodic_flushLockrsuper__init__DD_MAX_BATCH_SIZE exceptionstr)selfkwargs dd_base_urle __class__s Y/var/www/openai/venv/lib/python3.12/site-packages/litellm/integrations/datadog/datadog.pyr+zDataDogLogger.__init__+sk#   !B Cyyt,4 KLLyyD)1 MNN 61AA!D !ii 5DO+BIIi,@+AN O -.,99/0,99]+  &%0M">02D     3 3 5 6%llnDO G   %)__AR    $ $@QI G  sE*E.. F7#FFcK tjd||j||||d{y7#t$r@}tjdt |dt jYd}~yd}~wwxYww)aF Async Log success events to Datadog - Creates a Datadog payload - Adds the Payload to the in memory logs queue - Payload is flushed every 10 seconds or when batch size is greater than 100 Raises: Raises a NON Blocking verbose_logger.exception if an error occurs 7Datadog: Logging - Enters logging function for model %sNDatadog Layer Error -  r r_log_async_eventr!r-r. traceback format_excr/r0 response_obj start_timeend_timer2s r4async_log_success_eventz%DataDogLogger.async_log_success_event[sz   I6 '' j(S S S   $ $(Q93G3G3I2JK    7B-979B9 B6A=8B=BBcK tjd||j||||d{y7#t$r@}tjdt |dt jYd}~yd}~wwxYww)Nr6r7r8r9r=s r4async_log_failure_eventz%DataDogLogger.async_log_failure_eventssz   I6 '' j(S S S   $ $(Q93G3G3I2JK    rBcK |jstjdytjdt |j|j |j |jd{}|jdk(r.tjtjjy|j|jdk7r%td|jd|jtjd|j|jy7#t$r@}tjd t|d tj Yd}~yd}~wwxYww) z Sends the in memory logs queue to datadog api Logs sent to /api/v2/logs DD Ref: https://docs.datadoghq.com/api/latest/logs/ Raises: Raises a NON Blocking verbose_logger.exception if an error occurs z!Datadog: log_queue does not existNz(Datadog - about to flush %s events on %si'Response from datadog API status_code: , text:  >>(()LM  :DNN#  "<R>R=SS[\d\i\i[jk  N$$  M    $ $4SVHBy?S?S?U>VW   sYE8!D,E8AD,8D*9AD,9E8:A/D,)E8*D,, E556E0+E80E55E8c^ tjdur|j||||}n|j||||}|jj |j |d|ji}|j|jdk7r%td|jd|jtjd|j|jy #t$r@}tjd t|d t!j"Yd }~y d }~wwxYw) z Sync Log success events to Datadog - Creates a Datadog payload - instantly logs it on DD API Tr0r>r?r@ DD-API-KEY)urljsonheadersrFrGrHrIr7r8N)litellmdatadog_use_v1_create_v0_logging_payloadcreate_datadog_logging_payloadr%postr$rrQrMr!rRr rr-r.r;r<)r/r0r>r?r@ dd_payloadrSr2s r4log_success_eventzDataDogLogger.log_success_eventsF( %%-!<<!!-)% = "@@!!-)% A '',,OO $//-H  % % '##s*=h>R>R=SS[\d\i\i[jk  N$$      $ $(Q93G3G3I2JK    sC C## D,,6D''D,c,K|j||||}|jj|tjd|j dt |j|jk\r|jd{yy7w)NrVz-Datadog, event added to queue. Will flush in z seconds...) r^rJappendr rflush_intervalrKrrT)r/r0r>r?r@r`s r4r:zDataDogLogger._log_async_events88%! 9  j);Dr?r@returncddl}|jdd}| tdtj}|jddk(rtj }|j ||j|t}tjd|t|j|j|j||j| } | S) a Helper function to create a datadog payload for logging Args: kwargs (Union[dict, Any]): request kwargs response_obj (Any): llm api response start_time (datetime.datetime): start time of request end_time (datetime.datetime): end time of request Returns: DatadogPayload: defined in types.py rNstandard_logging_objectz+standard_logging_object not found in kwargsstatusfailuredefault&Datadog: Logger - Logging payload = %sddsourceddtagshostnamemessageservicerh)rYget ValueError DataDogStatusINFOERROR)truncate_standard_logging_payload_contentdumpsr.r rDatadogPayload_get_datadog_source_get_datadog_tags_get_datadog_hostname_get_datadog_service) r/r0r>r?r@rYrgrh json_payloadr`s r4r^z,DataDogLogger.create_datadog_logging_payloads& DJJJ %tE  # *JK K## " & &x 0I ="((F 667NOzz"93zG E|T#--/))+//1 --/  datacKddl}ddl}|j|j|tj d}|j j|j||jdddd{}|S7w) a; Async helper to send compressed data to datadog self.intake_url Datadog recommends using gzip to compress data https://docs.datadoghq.com/api/latest/logs/ "Datadog recommends sending your logs compressed. Add the Content-Encoding: gzip header to the request when sending" rNrjzutf-8gzipzapplication/json)rWzContent-Encodingz Content-Type)rXrrZ) rrYcompressryr.encoder#r_r$r)r/rrrYcompressed_datarSs r4rLz(DataDogLogger.async_send_compressed_datas| -- 4 (E(L(LW(UV**// "oo$* 20   sA7B9B:Bpayloaderrorparent_otel_spanevent_metadatac4K ddl}|j}|j|t} t ddd| dt j } |jj| y#t$r"} tjd| Yd} ~ yd} ~ wwxYww) z Logs failures from Redis, Postgres (Adjacent systems), as 'WARNING' on DataDog - example - Redis is failing / erroring, will be logged on DataDog rNrjr[litellm-serverrmz;Datadog: Logger - Exception in async_service_failure_hook: ) rY model_dumpryr.rzruWARNrJrcr!r r-) r/rrrr?r@rrY _payload_dict_dd_message_str _dd_payloadr2s r4async_service_failure_hookz(DataDogLogger.async_service_failure_hook5s  #..0M"jjjDO("'($)) K NN ! !+ .    $ $MaSQ    s/BA%A*)B* B3B BBBc Kyw)z Logs success from Redis, Postgres (Adjacent systems), as 'INFO' on DataDog No user has asked for this so far, this might be spammy on datatdog. If need arises we can implement this Nr)r/rrrr?r@rs r4async_service_success_hookz(DataDogLogger.async_service_success_hookZs  scddl}|jdi}|jdixsi}|jd}|jdi} |jdd} |jd d } |d } |jd ttj} t | } ||z j d z} t |}i}t|t r"|jD]\}}|dvr |||<| | | ||||jdd|jdd| |jdd||| |d}|j|t}tjd|t|j|j|j!||j#t$j&}|S#t$rd}YwxYw#t$r|}YwxYw)z Note: This is our V1 Version of DataDog Logging Payload (Not Recommended) If you want this to get logged set `litellm.datadog_use_v1 = True` rNlitellm_paramsmetadatamessagesoptional_params call_typezlitellm.completion cache_hitFusageidr)endpointcaching_groupsprevious_modelsmodelruser response_cost)rrrr?r@ response_timerrmodel_parametersspendrrSrrrjrlrm)rYrsr.uuiduuid4dict total_secondsr! isinstanceitemsryr rrzr{r|r}r~rurv)r/r0r>r?r@rYrrrrrrrrrclean_metadatakeyrPrrr`s r4r]z(DataDogLogger._create_v0_logging_payloadjs $4b9   z2 . 4" ::j) **%6;JJ{,@A JJ{E2 W%   dC $5 6U  !% 2AACdJM ( -L h %&nn. U */N3'/""$ *ZZ,JJvr* /ZZ3 $& "zz'3z7 E|T#--/))+//1 --/ %%  m ! M !  ('L (s$2G G! GG! G0/G0c dtjdddtjdddtjddd tjd tjd d S) Nzenv:DD_ENVunknownz ,service: DD_SERVICEr[z ,version: DD_VERSIONz ,HOSTNAME:z ,POD_NAME:POD_NAME)rr rr}rrr4r|zDataDogLogger._get_datadog_tagssbii)45YryyW`?a>bbklnluluwCENmOlPPZ[h[~[~[@ZAAKLNLULUV`bkLlKmn nrc.tjddS)N DD_SOURCEr[rr rrr4r{z!DataDogLogger._get_datadog_sourcesyyi00rc.tjddS)Nrrrrrr4r~z"DataDogLogger._get_datadog_servicesyy'788rc.tjddS)NHOSTNAMErrrrr4r}z#DataDogLogger._get_datadog_hostnamesyyR((rc.tjddS)Nrrrrrr4_get_datadog_envzDataDogLogger._get_datadog_envsyy9--rc.tjddS)Nrrrrrr4_get_datadog_pod_namez#DataDogLogger._get_datadog_pod_namesyyY//r)rNNNN)"__name__ __module__ __qualname__r+rArDrTrar:rrrrrzr^rrrLrrr. datetimeObjfloatrrr] staticmethodr|r{r~r}rr __classcell__)r3s@r4rr)sB.`0 )V0 d*"-dCi --%% - ## -  -^Th6 "*.:>8<)-# %# }# #3- # U;#567 # 5 !345 # !# P "*.:>8<)-%}#3-  U;#567  5 !345 ! RdCi RR%% R ## R  Rhnn1199))..00rr)__doc__r&rrr;rrtypingrrrrhttpxrr[litellm._loggingr (litellm.integrations.custom_batch_loggerr &litellm.llms.custom_httpx.http_handlerr r r "litellm.types.integrations.datadoglitellm.types.servicesrlitellm.types.utilsrr,rrrr4rsZ   ,--+F 176k0%k0r