$5 free Limited time. Sign up to activate. Get your $5 →
ArticleData feeds

Professional profile dataset: fields, delivery and license

One record per person, delivered as files to your bucket or as a Snowflake share. This is what sits inside a record, how the files arrive, what the dates mean and what to ask before you sign.

Written by Justin Winter, Head of Growth Founder and growth executive across VC-backed startups and B2B SaaS.
On this page

Key takeaways

A professional profile dataset is one record per person: identity, location, current and past roles, education, skills and certifications, with the date each record was last checked. It is licensed annually and delivered as compressed files to your S3 or GCS bucket, or as a Snowflake private listing.

  • Every record carries date_updated (the last crawl) and schema_version (the shape of the payload). Read both before you load a row.
  • Delivery cadence (daily, weekly, monthly) is a separate setting from how often records are recrawled.
  • Display, resale and retention terms are contract questions. Ask them before the sample run, not after.

What the search turns up

Search for a profile dataset and page 1 is two Reddit r/datasets threads, a Kaggle notebook, a 2022 post about a dataset product that no longer exists, a vendor's GitHub sample, a ResearchGate figure and a jobs scraper. Three different jobs share one results page.

A research sample is a fixed file from one point in time. The fields are whatever the author kept, and the license is rarely stated. The GitHub sample at #4 lists company fields only, gives two different record counts on the same page and says nothing about terms (retrieved 2026-11-01). Fine for a notebook. Not something you ship on.

A scraped file you build yourself gives you the fields you choose and the maintenance that comes with them: breakage when the page changes, blocking, and the legal exposure of the collection method.

A licensed feed has a published schema, a delivery schedule, written terms and records that get recrawled over time. That is the one a product team can build on, and it is the one this article covers, using Forager's published person schema as the worked example.

Evidence
850M+

person and company records in one dataset.

Source: Forager

Anatomy of one record

Anatomy of one person record in a data feed: field groups for identity, location, roles, education, skills and more, plus date_updated and schema_version; delivered as NDJSON, CSV or Parquet, gzip-compressed, to Amazon S3, Google Cloud or a Snowflake share.

The person schema (version 2.0.4) has 35 top-level fields. Two are required: forager_id and date_updated. Everything else is optional, so a loader has to accept nulls and empty arrays on every other field. Grouped by what they are for:

GroupFieldsNotes
Identityforager_id, linkedin_id, linkedin_url, linkedin_slug, first_name, last_name, headline, description, photoforager_id is the stable key. A small subset of records has no profile identifier, usually because the profile was banned.
Locationcountry, area, city, location, locations[], linkedin_country, linkedin_areaEach entry in locations[] has a name and a place_type.
Rolesroles[]Each role: role_title, start_date, end_date, employment_type, organization_id, organization_name, organization_domain, organization_industry, location.
Educationeducations[], courses[], test_scores[]school_name is required on an education entry; degree, field_of_study, grade and dates are optional.
Credentialscertifications[], languages[], honors[], patents[], publications[], projects[]A certification carries certificate_name (required), authority, certificate_id, certificate_url and dates.
Profile signalsskills[], industry, number_of_connections, number_of_followers, is_available, volunteering[], volunteer_causes[], organizations[]skills is a plain array of strings.
Freshness and versiondate_updated (in the record) plus schema_version (on the row)Covered in the next two sections.

Two details in roles[] matter for anyone building "current employer":

  • There is no is_current flag. A role with no end_date is the current one. A person can have more than one.
  • employment_type is a numeric code, not a label. 12 is full_time, 2 is contract, 18 is internship; the full code table is on the Role schema page.

The full dictionary, with every sub-object, is at data-license/v2/schema.

How it arrives

Each row in a delivered file is a wrapper around the record:

FieldWhat it holds
idForager record id (integer)
date_updatedWhen the record was last crawled, UTC
schema_versionVersion of the payload, for example 2.0.4
serialized_dataThe record itself, shaped by the schema for that feed type and version

Files in a bucket. NDJSON is the default; CSV, TSV (any delimiter) and Parquet are also supported. Files are UTF-8, partitioned and compressed with gzip, bz2, brotli or zstd. The path tells you what is inside before you open it:

/forager_data_feed/<data_feed_id>/<date_created>/<data_feed_type>_<schema_version>/data_<partition_id>.<file_type>.<compression_type>

/forager_data_feed/123/2025-10-15/person_2.0.4/data_0_0_1.json.gzip

data_feed_type is person, organization or job. The schema version sits in the directory name, so a version change shows up in the path before it shows up in a row.

Drops go to an Amazon S3 bucket you own (the preferred route: create the bucket, attach the documented IAM policy to a user for Forager, send the keys) or to Google Cloud Storage. Azure Blob is coming soon. Cadence is daily, weekly or monthly.

Snowflake private listing. Request the Forager.ai listing for each feed in your license on Snowflake Marketplace, then pick it up under Data Products, Private Sharing, with a role that has CREATE DATABASE and IMPORT SHARE. The feed appears as a database you query like any internal table.

Net-new transfers. A transfer can be limited to records crawled since the last scheduled transfer, so a weekly drop is the week's recrawls rather than the whole dataset again. Ask which mode your feed uses.

Docs: output file details, S3, GCS, Snowflake.

Reading freshness

date_updated is the moment the record was last crawled. It is the only freshness fact in the file, so three things follow from it.

The total count and the updated count differ. A feed's total is every record currently in the dataset. The "updated in the last month" count is how many of those were recrawled in that window. The gap is expected. Records stop being crawled when a profile goes inactive, is banned or no longer exists, and a record that is not crawled is not transferred again. So the records that stop moving are the ones that stopped existing.

Delivery cadence is not refresh cadence. Daily, weekly or monthly controls how often recrawled records reach you. It says nothing about how often the underlying data is recrawled. Forager's published line is that records are recrawled on a rolling basis; the refresh term belongs in your agreement, in writing. Any vendor's cadence claim should be a contract term, and one you can check against date_updated yourself.

Measure it on your side. Plot the distribution of date_updated across the records your product cares about, then plot it again next month. That curve is the refresh rate as it applies to you, not as it applies to the whole dataset. B2B data decay: how to measure it on your own list walks through the method.

Joining the three feeds

A license can cover three feeds: person, organization and job post. They join on organization keys.

  • A person's roles[] entry carries organization_id, organization_name and organization_domain.
  • An organization record carries forager_id, domain, name, industry, employees_range, country and city.
  • A job post carries organization_forager_id and organization_domain, plus title, url, date_featured, is_active and is_remote.

The documented recipe flattens roles, keeps the ones with no end_date, lowercases the domain and joins it to the organization feed:

CREATE OR REPLACE VIEW CURRENT_ROLES AS
SELECT
  p.serialized_data:forager_id::INTEGER        AS person_id,
  r.value:role_title::STRING                   AS role_title,
  LOWER(r.value:organization_domain::STRING)   AS organization_domain
FROM PERSON_FEED p,
     LATERAL FLATTEN(input => p.serialized_data:roles) r
WHERE r.value:end_date IS NULL
  AND r.value:organization_domain IS NOT NULL;

Then JOIN ORGANIZATION_FEED o ON LOWER(o.serialized_data:domain::STRING) = organization_domain. To add job posts, join organization_forager_id to the organization's forager_id. Watch for people with two open roles (they appear twice), organizations that share a domain, and roles with no domain at all (count them so you know how many drop out). On job posts date_updated can be null; on person and organization records it is always set.

Full recipe: Join the people and company feeds in Snowflake.

License questions to ask any vendor

A sample file tells you what the data looks like. The agreement tells you what you may do with it. Six questions worth asking before the sample run, with Forager's answers where they are published:

  1. Can my users see it? Whether records, or fields from them, can be displayed inside your product to your own customers. Forager: agreed in the license agreement.
  2. Can I pass it on? Whether you may resell, sublicense or redistribute records or derived datasets. Forager: agreed in the license agreement.
  3. What happens at the end of the term? Whether delivered copies must be deleted, and whether models or indexes trained on them count. Forager: set in the license agreement; ask before the sample run.
  4. How are removals handled? What the vendor does when a record is no longer valid or legally distributable, and what you are expected to do with copies already delivered. Forager stops crawling the record and it is not transferred again.
  5. Whole dataset or a subset? Whether the license can be scoped to the regions, industries or roles you need. Forager: annual license, the entire dataset or a custom subset.
  6. How do schema changes reach me? Forager versions the schema; a new version is released when fields are added or structures change, changes are designed to be additive wherever possible, and the version is in every file path and every row, so you migrate on your own timeline.

Evaluate with a sample on your own universe

Record counts describe the dataset. Fill rate describes the dataset where you need it. Measure the second one yourself:

  1. Take the people your product already knows about, not a list the vendor picks: a few thousand linkedin_slug or linkedin_url values from your own tables.
  2. Match them against the sample and count the hits.
  3. For the hits, count fill per field: a current role (end_date null), a non-empty educations[], a skills[] with more than a few entries, a city.
  4. Look at the date_updated distribution for those records, not for the sample as a whole.
  5. Repeat on a second sample a month later if you can. The change tells you more than either snapshot.

The numbers that come out are yours. We publish no fill rate, because the only one that matters is the one on your records.

FAQ

What fields are in a person record?

Identity (forager_id, network ID, profile URL and slug, names, headline, description, photo), location (country, area, city, location, locations[]), roles[], educations[], certifications[], skills[], languages[], projects[], honors[], patents[], publications[], courses[], test_scores[], volunteering[], organizations[], follower and connection counts, industry, is_available and date_updated. Only forager_id and date_updated are required.

How is the dataset delivered?

As compressed files (NDJSON by default; CSV, TSV or Parquet) dropped daily, weekly or monthly into your own S3 or GCS bucket, or as a Snowflake private listing you query in place. Azure Blob is coming soon.

What does date_updated mean?

The time the record was last crawled, in UTC. It appears on the row wrapper and inside the record. Records that are no longer crawled keep their last value and are not transferred again.

Can I receive only new or changed records?

Yes. A transfer can be limited to records crawled since the last scheduled transfer.

Can I display or resell the data?

Both are license terms, agreed per contract. Ask for them in writing before you build on a sample.

Bring the identifiers you already hold

Person, company and job-post feeds across 800M+ people and 65M+ companies, delivered to your warehouse on an annual license. Bring the identifiers you already hold and we will run the sample.

  • 800M+ people and 65M+ companies
  • Mobile and email lookups that find nothing cost 0 credits
  • Free API key to start

#1 of 10 in Clay's 2026 test of mobile data providers Source: Clay

Justin Winter
Justin Winter Head of Growth at Forager. Founder and growth executive across VC-backed startups and B2B SaaS.
Start building

Put people, company and job data your users can act on inside your product

Send us a list. We run it, show you what we find, then talk volume and terms.

  • $5 free credits
  • No credit card
  • Run your sample
Platforms and teams that build on Forager
  • Clay
  • Juicebox
  • Nooks
  • Andela
  • HubSpot
  • SeekOut