g'rddlZddlZddlZddlZddlmZmZmZmZddl Z ddl m Z ddl m Z mZddlmZddlmZddlmZddlmZmZ d&d ed ed d eefd Zded ed d edejdejddf dZd edededejdejddf dZ d'deed ed defdZ d'ded ed deefdZ de!deefdZ" d'deed ed defdZ# d'deed ed defdZ$dedefdZ%dedefd Z&dede'fd!Z(d"edejdejd ed#ed$edefd%Z)y)(N)AnyListLiteralOptional)verbose_logger)"BATCH_STATUS_POLL_INTERVAL_SECONDSBATCH_STATUS_POLL_MAX_ATTEMPTS) afile_content)Logging)Batch)StandardLoggingPayloadUsagebatch_idcustom_llm_provider)openaiazure vertex_ai logging_objc zKddlm}tjd| t dt t D]} tjj}|||d{}tjd|j|jdk(r9tjj}td |||||d|d{y|jd k(r tjtd{y77<#t$r } tjd | Yd} ~ Ld} ~ wwxYw77w) z Async Job waits for the batch to complete and then logs the completed batch usage - cost, total tokens, prompt tokens, completion tokens Polls retrieve_batch until it returns a batch with status "completed" or "failed" )aretrieve_batchzF.....in _batches_async_logging... polling retrieve to get batch statusNzDlogging_obj is None cannot calculate cost / log batch creation eventz-in _batches_async_logging... batch status= %s completed)batchrr start_timeend_timefailedzerror in batches_async_logging)mainrrdebug ValueErrorranger datetimenowstatus_handle_completed_batch Exceptionerrorasynciosleepr) rrrkwargsr_rrres P/var/www/openai/venv/lib/python3.12/site-packages/litellm/batches/batch_utils.pybatches_async_loggingr.s9&P R  1 2 F!**..0J!0;N!OOE  ? ||{*#,,002-(; +)%   )mm>???/3P F  !A1 E E F?sk;D;*D (D )A%D D D D;D %D;D9D; D  D  D6D1,D;1D66D;rrrreturnc Kt||d{}t||d{}t||}td|||||d|d{y7@7-7 w)z?Helper function to process a completed batch and handle loggingN)rfile_content_dictionaryr1r)r batch_usage batch_costrrr),_get_batch_output_file_content_as_dictionary_batch_cost_calculator,_get_batch_job_total_usage_from_file_content_log_completed_batch) rrrrrr*r1r4r3s r-r%r%Bs%Q "% ./ 7J? 7/K        s1AAAA%A AAAAr3r4cKd|_t||||||}||jd<tj|j d||dt j|jd||fjyw)zFHelper function to handle all logging operations for a completed batch batch_success)r*rrrbatch_usage_object response_coststandard_logging_objectN)resultrr cache_hit)targetargs) call_type3_create_standard_logging_object_for_completed_batchmodel_call_detailsr( create_taskasync_success_handler threadingThreadsuccess_handlerstart)rr3r4rrr*r=s r-r8r8es,KQ& AXK""#<= ))! * **J ) egsBBr1ctK|dk(r tdt||}tjd||Sw)zC Calculate the cost of a batch based on the output file id r1Vertex AI does not support file content retrievalr2 total_cost=%s)r %_get_batch_job_cost_from_file_contentrr)r1r total_costs r-r6r6sDk)LMM6 7/J*5 s68cK|dk(r td|j tdt|j|d{}t|jS7w)zE Get the batch output file content as a list of dictionaries rrLNz3Output file id is None cannot retrieve file content)file_idr)r output_file_idr _get_file_content_as_dictionarycontent)rr _file_contents r-r5r5sgk)LMM #NOO'$$/M +=+@+@ AA sAA AA  file_contentc@ |jd}g}|jjdD])}|s|jt j |+t jdt j|d|S#t$r}|d}~wwxYw)zO Get the file content as a list of dictionaries from JSON Lines format zutf-8 zjson_objects=%sindentN) decodestripsplitappendjsonloadsrrdumpsr&)rV_file_content_str json_objectsliner,s r-rSrSs (//8 %++-33D9D##DJJt$45: . A?? B%B  B%cd}d}d}|D]Q}t|st|}t|}||jz }||jz }||j z }St |||S)z= Get the tokens of a batch job from the file content r) total_tokens prompt_tokenscompletion_tokens)rhri'_get_batch_job_usage_from_response_bodyrorprqr)r1rrorprqrlrmusages r-r7r7sLM( )% 0EeLNB>RE E.. .L U00 0M !8!8 8  ) !#+  response_bodycH|jddxsi}tdi|}|S)z> Get the tokens of a batch job from the response body rsNr)getr)ru _usage_dictrss r-rrrrs- ##GT28bK';'E Lrtbatch_job_output_filec^|jddxsi}|jddxsi}|S)z9 Get the response from the batch job output file responseNbodyrw)ry _responserms r-riris6,// DAGRI]]6406BN rtcX|jddxsi}|jdddk(S)z7 Check if the batch job response status == 200 r{N status_coder})ryr~s r-rhrhs1,// DAGRI == - 44rtr*r;r<c|jjdd}| tdd|d<||d<|j|d<|j|d<|j |d <|S) z@ Create a standard logging object for a completed batch r=Nzrsk //+-S+= DL/3,@,@ !?@,@+,,@^    !?@ # !!     F"""""!! "  " "NDL!$Z !?@ &DLB B !?@B $ZB(%DJ&DL!$Z !?@ :DL!$Z !?@ 04EDS5$545# #!!### #  #  ##rt