Current Path: > > opt > alt > python311 > lib64 > python3.11 > > urllib > __pycache__
Operation : Linux premium131.web-hosting.com 4.18.0-553.44.1.lve.el8.x86_64 #1 SMP Thu Mar 13 14:29:12 UTC 2025 x86_64 Software : Apache Server IP : 162.0.232.56 | Your IP: 216.73.216.111 Domains : 1034 Domain(s) Permission : [ 0755 ]
Name | Type | Size | Last Modified | Actions |
---|---|---|---|---|
__init__.cpython-311.opt-1.pyc | File | 163 bytes | June 23 2025 15:47:46. | |
__init__.cpython-311.opt-2.pyc | File | 163 bytes | June 23 2025 15:47:46. | |
__init__.cpython-311.pyc | File | 163 bytes | June 23 2025 15:47:46. | |
error.cpython-311.opt-1.pyc | File | 3898 bytes | June 23 2025 15:47:46. | |
error.cpython-311.opt-2.pyc | File | 3224 bytes | June 23 2025 15:47:55. | |
error.cpython-311.pyc | File | 3898 bytes | June 23 2025 15:47:46. | |
parse.cpython-311.opt-1.pyc | File | 55546 bytes | June 23 2025 15:47:46. | |
parse.cpython-311.opt-2.pyc | File | 44959 bytes | June 23 2025 15:47:55. | |
parse.cpython-311.pyc | File | 55546 bytes | June 23 2025 15:47:46. | |
request.cpython-311.opt-1.pyc | File | 126343 bytes | June 23 2025 15:47:50. | |
request.cpython-311.opt-2.pyc | File | 114349 bytes | June 23 2025 15:47:55. | |
request.cpython-311.pyc | File | 126562 bytes | June 23 2025 15:47:46. | |
response.cpython-311.opt-1.pyc | File | 5206 bytes | June 23 2025 15:47:46. | |
response.cpython-311.opt-2.pyc | File | 4630 bytes | June 23 2025 15:47:55. | |
response.cpython-311.pyc | File | 5206 bytes | June 23 2025 15:47:46. | |
robotparser.cpython-311.opt-1.pyc | File | 13347 bytes | June 23 2025 15:47:46. | |
robotparser.cpython-311.opt-2.pyc | File | 12023 bytes | June 23 2025 15:47:55. | |
robotparser.cpython-311.pyc | File | 13347 bytes | June 23 2025 15:47:46. |
� !A?h�$ � � � d Z ddlZddlZddlZdgZ ej dd� � Z G d� d� � Z G d� d� � Z G d � d � � Z dS )a% robotparser.py Copyright (C) 2000 Bastian Kleineidam You can choose between two licenses when using this package: 1) GNU GPLv2 2) PSF license for Python 2.2 The robots.txt Exclusion Protocol is implemented as specified in http://www.robotstxt.org/norobots-rfc.txt � N�RobotFileParser�RequestRatezrequests secondsc �\ � e Zd ZdZdd�Zd� Zd� Zd� Zd� Zd� Z d � Z d � Zd� Zd� Z d � Zd� ZdS )r zs This class provides a set of methods to read, parse and answer questions about a single robots.txt file. � c � � g | _ g | _ d | _ d| _ d| _ | � |� � d| _ d S )NFr )�entries�sitemaps� default_entry�disallow_all� allow_all�set_url�last_checked��self�urls �9/opt/alt/python311/lib64/python3.11/urllib/robotparser.py�__init__zRobotFileParser.__init__ sG � ������ �!���!���������S��������� c � � | j S )z�Returns the time the robots.txt file was last fetched. This is useful for long-running web spiders that need to check for new robots.txt files periodically. )r �r s r �mtimezRobotFileParser.mtime% s � � � � r c �@ � ddl }|� � � | _ dS )zYSets the time the robots.txt file was last fetched to the current time. r N)�timer )r r s r �modifiedzRobotFileParser.modified. s# � � ���� �I�I�K�K����r c �| � || _ t j � |� � dd� \ | _ | _ dS )z,Sets the URL referring to a robots.txt file.� � N)r �urllib�parse�urlparse�host�pathr s r r zRobotFileParser.set_url6 s4 � ����%�|�4�4�S�9�9�!�A�#�>��� �4�9�9�9r c � � t j � | j � � }|� � � }| � |� d� � � � � � � dS # t j j $ rK}|j dv rd| _ n)|j dk r|j dk rd| _ Y d}~dS Y d}~dS Y d}~dS Y d}~dS d}~ww xY w)z4Reads the robots.txt URL and feeds it to the parser.zutf-8)i� i� Ti� i� N) r �request�urlopenr �readr �decode� splitlines�error� HTTPError�coder r )r �f�raw�errs r r&