Using Network Client

The secedgar.client.NetworkClient class is used to interact with the SEC’s website. This class aims to provide all utilities needed to make requesting data from the SEC as easily as possible.

Note

By default, secedgar will use NetworkClient. Modifying NetworkClient is not necessary unless you need further control over how you would like to interact with the SEC’s website.

class secedgar.client.NetworkClient(user_agent, retry_count=3, batch_size=10, backoff_factor=0, rate_limit=10)

Class in charge of sending and handling requests to EDGAR database.

Args: user_agent (str): Value used for HTTP header “User-Agent” for all requests.

Must be given. See the SEC’s statement on fair access for more information.

retry_count (int, optional): Number of times to retry connecting to URL if not successful.

Defaults to 3.

batch_size (int, optional): Number of filings to receive per request

Helpful if pagination needed. Defaults to 10.

backoff_factor (float, optional): Backoff factor to use with urllib3.util.retry.Retry.

See urllib3 docs for more info. Defaults to 0.

rate_limit (int, optional): Number of requests per second to limit to.

Defaults to 10.

Examples

Creating a basic instance of NetworkClient is straightforward. The only required argument is the user_agent argument.

from secedgar.client import NetworkClient
client = NetworkClient(user_agent="Name (email)")

If you are running into 429 errors, you may want to consider modifying your backoff_factor.

Note

By default, you can pass any keyword arguments you want to give NetworkClient to any of the various filings classes. The secedgar.filings() function also accepts any/all of the NetworkClient keyword arguments.

from secedgar.client import NetworkClient
client = NetworkClient(user_agent="Name (email)", backoff_factor=1)
property backoff_factor

Backoff factor to pass to urllib3.util.retry.Retry.

Type:

float

property batch_size

The Number of results to show per page.

Type:

int

async static fetch(link, session)

Asynchronous get request.

Parameters:

  • link (str) – URL to fetch.

  • session (aiohttp.ClientSession) – Asynchronous client session to use to perform get request.

Returns:

Contents of response from get request.

Return type:

Content

get_response(path, params=None, **kwargs)

Execute HTTP request and returns response if valid.

Parameters:

  • path (str) – A properly-formatted path

  • params (dict) – Dictionary of parameters to pass to request. Defaults to None.

  • backoff_factor (float) – Backoff factor to pass to urllib3.util.Retry.

  • kwargs – Keyword arguments to pass to requests.Session.get.

Returns:

A requests.Response object.

Return type:

response (requests.Response)

Raises:

EDGARQueryError – If problems arise when making query.

get_soup(path, params, **kwargs)

Return BeautifulSoup object from response text. Uses lxml parser.

Parameters:

  • path (str) – A properly-formatted path

  • params (dict) – Dictionary of parameters to pass to request.

Returns:

BeautifulSoup object from response text.

property rate_limit

Number of requests to limit client to per second.

Type:

int

property retry_count

Number of times to retry request.

Type:

int

property user_agent

Value used for HTTP header “User-Agent” for all requests.

Type:

str

async wait_for_download_async(inputs)

Asynchronously download links into files using rate limit.

Parameters:

inputs (list of tuples of str) – List of tuples with length 2. First element in tuple should be URL to request and second element should be path where content after requesting URL is stored.