g'-ddlZddlmZmZmZmZmZddlZGddZe dk(reZ e e je de e jdd e d e e jd e d e e jd e de e jdddd e de e jdddyy)N)OptionalLiteralListDictTuplec eZdZdZdZddedeefdZ dde e dd e ed e ed e e d dedef d Z de eeffdZ dZdedeeeffdZdedefdZy)UserAgentGeneratora| Generate random user agents with specified constraints. Attributes: desktop_platforms (dict): A dictionary of possible desktop platforms and their corresponding user agent strings. mobile_platforms (dict): A dictionary of possible mobile platforms and their corresponding user agent strings. browser_combinations (dict): A dictionary of possible browser combinations and their corresponding user agent strings. rendering_engines (dict): A dictionary of possible rendering engines and their corresponding user agent strings. chrome_versions (list): A list of possible Chrome browser versions. firefox_versions (list): A list of possible Firefox browser versions. edge_versions (list): A list of possible Edge browser versions. safari_versions (list): A list of possible Safari browser versions. ios_versions (list): A list of possible iOS browser versions. android_versions (list): A list of possible Android browser versions. Methods: generate_user_agent( platform: Literal["desktop", "mobile"] = "desktop", browser: str = "chrome", rendering_engine: str = "chrome_webkit", chrome_version: Optional[str] = None, firefox_version: Optional[str] = None, edge_version: Optional[str] = None, safari_version: Optional[str] = None, ios_version: Optional[str] = None, android_version: Optional[str] = None ): Generates a random user agent string based on the specified parameters. cddddddddd d d |_d d ddddddd|_dgdgdgdggddgddgddgggdgdgd|_ddgd d!|_gd"|_gd#|_gd$|_gd%|_y)&Nz(Windows NT 10.0; Win64; x64)z(Windows NT 10.0; WOW64))10_6410_32z#(Macintosh; Intel Mac OS X 10_15_7)z+(Macintosh; Intel Mac OS X 10.15; rv:109.0))intelnewerz(X11; Linux x86_64)z(X11; Ubuntu; Linux x86_64)z(X11; CrOS x86_64 14541.0.0))genericubuntu chrome_os)windowsmacoslinuxz(Linux; Android 13; SM-S901B)z(Linux; Android 12; Pixel 6)z"(Linux; Android 13; OnePlus 9 Pro)z(Linux; Android 12; M2102J20SG))samsungpixeloneplusxiaomiz*(iPhone; CPU iPhone OS 16_5 like Mac OS X)z!(iPad; CPU OS 16_5 like Mac OS X))iphoneipad)androidioschromefirefoxsafariedgegeckowebkit)rrr )r"rr)zAppleWebKit/537.36zAppleWebKit/605.1.15)Gecko/20100101r&z Gecko/2010010) chrome_webkit safari_webkitr!)zChrome/119.0.6045.199zChrome/118.0.5993.117zChrome/117.0.5938.149zChrome/116.0.5845.187zChrome/115.0.5790.171)zEdg/119.0.2151.97zEdg/118.0.2088.76zEdg/117.0.2045.47zEdg/116.0.1938.81zEdg/115.0.1901.203)z Safari/537.36zSafari/605.1.15z Safari/604.1z Safari/602.1zSafari/601.5.17) z Firefox/119.0zFirefox/118.0.2zFirefox/117.0.1z Firefox/116.0zFirefox/115.0.3zFirefox/114.0.2zFirefox/113.0.1z Firefox/112.0zFirefox/111.0.1z Firefox/110.0)desktop_platformsmobile_platformsbrowser_combinationsrendering_engineschrome_versions edge_versionssafari_versionsfirefox_versions)selfs R/var/www/openai/venv/lib/python3.12/site-packages/crawl4ai/user_agent_generator.py__init__zUserAgentGenerator.__init__#s93 ?F 17;" $;7?; G; !      )$8$8$ -.% !(23"     !  num_browsersreturnc||jvrtd|tj|j|}g}|D]0}|dk(r/|j tj|j 8|dk(r/|j tj|j l|dk(r/|j tj|j|dk(r/|j tj|j|dk(r3|j tj|jd |dk(s|j |jd3|S) a Get a valid combination of browser versions. How it works: 1. Check if the number of browsers is supported. 2. Randomly choose a combination of browsers. 3. Iterate through the combination and add browser versions. 4. Return the browser stack. Args: num_browsers: Number of browser specifications (1-3) Returns: List[str]: A list of browser versions. z Unsupported number of browsers: rrrr r!r"r') r+ ValueErrorrandomchoiceappendr-r0r/r.r,)r1r5 combination browser_stackbrowsers r2get_browser_stackz$UserAgentGenerator.get_browser_stacks4 t88 8? ~NO OmmD$=$=l$KL  "G("$$V]]43G3G%HII%$$V]]43H3H%IJH$$$V]]43G3G%HIF"$$V]]43E3E%FGG#$$V]]43I3I'3R%STH$$$T%;%;O%LM#r4N device_type)desktopmobileos_type device_brand browser_typerr rrc|j|||}d|g}|j|}dt|vr2|jt j |j dnIdt|vs dt|vr/|j|j d|jd|j|dj|S) a Generate a random user agent with specified constraints. Args: device_type: 'desktop' or 'mobile' os_type: 'windows', 'macos', 'linux', 'android', 'ios' device_brand: Specific device brand browser_type: 'chrome', 'edge', 'safari', or 'firefox' num_browsers: Number of browser specifications (1-3) z Mozilla/5.0Firefoxr!ChromeSafarir'z(KHTML, like Gecko) ) get_random_platformr?strr;r9r:r,extendjoin) r1r@rCrDrEr5platform componentsr=s r2generatezUserAgentGenerator.generates"++K,O$X. ..|<  M* *   fmmD,B,B7,KL M ]+ +x3};M/M   d44_E F   3 4 -(xx ##r4c P|jdi|}|j|}||fS)z2Generate both user agent and matching client hints)rRgenerate_client_hints)r1kwargs user_agent client_hintss r2generate_with_client_hintsz-UserAgentGenerator.generate_with_client_hintss0"T]],V, 11*= <''r4c|dk(r |jn*|dk(r |jni|j|j}|r*|j|jfD]}||vs|||i}ntjt |j }|r|||vr|||Stjt ||j S)z9Helper method to get random platform based on constraintsrArB)r)r*r9r:listkeysvalues)r1r@rCrD platformsplatform_groupos_keys r2rLz&UserAgentGenerator.get_random_platforms.9Y.FD**,78,C4((Fd,,F0E0EF  #'#9#94;P;P"Qn,!(.*A BI#R tINN$456 LIf,==V$\2 2}}T)F"3":":"<=>>r4rWcddddd}i}|jD]2\}}tj||}|s|jd||<4|S)zDParse a user agent string to extract browser and version informationz Chrome/(\d+)z Edg/(\d+)z Version/(\d+)z Firefox/(\d+)rFr#)itemsresearchgroup)r1rWbrowsersresultr>patternmatchs r2parse_user_agentz#UserAgentGenerator.parse_user_agents_& &'   ( 0 GWIIgz2E"'++a.w!1  r4ct|j|}g}d|vr_|jd|dd|jdd|vr|jd|ddnK|jd|ddn2d|vry d |vr)|jd |d d|jdd j|S) z:Generate Sec-CH-UA header value based on user agent stringrz"Chromium";v=""z"Not_A Brand";v="8"r z"Microsoft Edge";v="z"Google Chrome";v="rz""rz "Safari";v="z, )rjr;rO)r1rWrfhintss r2rUz(UserAgentGenerator.generate_client_hintss((4 x  LL>(8*<)=Q? @ LL. /! 3HV4D3EQGH 28H3E2FaHI ( "  ! LL<(:';1= > LL. /yyr4)r#)NNNNr%)__name__ __module__ __qualname____doc__r3intrrMr?rrrRrrYrLrrjrUrTr4r2r r s8b H$c$$s)$NGK)-.2Y]$% #$%g.A&BC#$!##$'sm#$'w/T'UV #$ " #$ +. #$J(eCHo( ?"34S>"   r4r __main__z Single browser (Chrome):r#r)r5rEz Two browsers (Gecko/Firefox):r$)r5z% Three browsers (Chrome/Safari/Edge):r%z Firefox on Linux:rArr)r@rCrEr5z Chrome/Safari/Edge on Windows:r)r@rCr5) r9typingrrrrrrcr rn generatorprintrRrTr4r2rws 77 N N b z"$I )    &' )  !(  CD +, )  !  ,- 23 )  !  ,-   )     ,- )    -r4