3.1. Accessing Data

In today’s world, data exists in countless forms and is generated constantly - from files and databases to APIs and live streams. However, accessing this data is not always straightforward.

Data can be:

• Publicly available, such as open datasets on the internet

• Privately hosted, requiring secure access through credentials or tokens

Depending on the source, the mode of access also varies. For example:

• Public datasets can often be accessed directly via URLs, public APIs, or be generated with web scraping.

• Private data may require authentication methods like API keys, OAuth tokens, or secure shell (SSH) connections to remote servers.

In this section, we will explore various data access methods in detail:

• 3.1.1. Internet Access
• 3.1.2. API Access
• 3.1.3. Remote Server Access
• 3.1.4. Scrapping
• 3.1.5. Streaming

Understanding these methods is critical for building a reliable and flexible data acquisition pipeline.