# HubSpot property retirement audit

## The problem

You have overlapping custom properties such as "demo date", "demo booked date" and "discovery date", and asking people which one they use is only a guess. This script reads your portal and ranks custom properties by how safe they look to retire, using fill rate, last written date and what wrote the values.

## How it works

For one object type (contacts, companies or deals) the script:

1. Lists properties with `GET /crm/v3/properties/{object}` and keeps only the ones where `hubspotDefined` is false.
2. Counts all records with an empty search (`POST /crm/v3/objects/{object}/search`).
3. For each property, runs a search with a `HAS_PROPERTY` filter and limit 1 to read the `total`. That is the fill count.
4. Fetches the most recently modified records that have a value (sorted by the object's last-modified property, newest first, `--sample-size` records).
5. For each sampled record calls `GET /crm/v3/objects/{object}/{id}?propertiesWithHistory=<name>` and collects `timestamp`, `sourceType`, `sourceId` and `sourceLabel` of each history entry.
6. Computes last written, write sources (counts by `sourceType`, such as CRM_UI, WORKFLOWS, INTEGRATION, IMPORT, FORM, API) and distinct source ids.
7. Scores each property and lists likely duplicate pairs by label.

Scoring rules (all in `score()` in `property_audit.py`):

- Any INTEGRATION or API write in the sample: **REVIEW**. Someone outside HubSpot may depend on it.
- Otherwise, no write in the sample for over 365 days, or no values at all: **RETIRE_CANDIDATE**.
- Otherwise, fill rate under 2% but written recently: **REVIEW**.
- Otherwise, **KEEP**.
- Has values but no history could be read: **REVIEW**.

Duplicate pairs: label similarity above 0.7 (Python `difflib`), a shared word that is not generic, or two date properties sharing "date".

The script is read-only. It sends GET requests and search POSTs (a read). It has no write mode, so there is no `--apply` flag.

## Setup

1. In HubSpot go to Settings > Integrations > Private Apps and create a private app. Give it read scopes only: CRM object read for the object you audit (for example `crm.objects.deals.read`) and the property schema read scope (for example `crm.schemas.deals.read`). Scope names change, so check the current HubSpot docs.
2. Copy the token and set it in your shell: `export HUBSPOT_TOKEN=...` (do not paste it into any file).
3. Download `property_audit.py` and the `sample/` folder into one directory. You need Python 3 only; there is nothing to install.
4. Run the offline demo first (see below) to see the output format.
5. Run it live: `python3 property_audit.py --object deals --sample-size 20 --out out`
6. Open `out/property_audit.md` (readable) or `out/property_audit.csv` (sortable). RETIRE_CANDIDATE rows come first, then REVIEW, then KEEP.
7. For every non-KEEP property, do the manual checks in the next section before you change anything.

Flags: `--object` (contacts, companies, deals), `--sample-size` (default 20), `--mock DIR`, `--out DIR`, `--base-url` (default `https://api.hubapi.com`), `--sleep` (seconds between live calls, default 0.25), `--now YYYY-MM-DD` (fixes today's date for repeatable runs).

## Manual checklist per property

The script cannot see dependencies. For each REVIEW or RETIRE_CANDIDATE property:

1. Open the property's "Used in" tab and the archive panel and check workflows, lists, forms and reports. Lists and reports that only use the field as a filter are easy to miss.
2. Export the values with record IDs.
3. Rename the property with a `DEPRECATED_` prefix.
4. Hide it from forms.
5. Wait one full reporting cycle.
6. Archive it rather than hard deleting.

If `write_sources` shows INTEGRATION or API, match the `source_labels` and `source_ids` columns against your connected apps and private apps, and ask whoever pays for the Zapier or middleware account.

## Test it offline

```
python3 property_audit.py --object deals --mock sample/ --out out --now 2026-11-05
python3 test_fix.py
```

Mock mode replaces only the HTTP transport with lookups in `sample/properties.json`, `sample/search_totals.json` and `sample/history.json`. Scoring, sorting and file writing are the same code as live mode. The sample data is fake ("Sample Co"). `test_fix.py` also fakes `urlopen` to check that a 429 response is retried. It exits 0 on pass. See `TEST_RESULTS.md` for the real output.

## Limits and what it does not cover

- **Not tested against the live HubSpot API.** Only the offline fixtures and a faked HTTP layer were run. Endpoint paths, request bodies and scope names come from the HubSpot v3 CRM API as I know it, so check the current docs. The base URL is a flag.
- **Dependencies are not checked.** Workflows, lists, forms, reports and scoring are not read. Use the "Used in" tab.
- **Sampled, not complete.** Last written and sources come from the newest-modified records that have a value. A record modified recently for another reason ranks above one whose property was written long ago, and a rare writer may not appear.
- **API cost.** One history request per sampled record per property. With 20 samples and 100 properties that is up to 2,000 requests plus searches. Lower `--sample-size` if needed. The script sleeps between calls and retries on 429 and 502/503/504, honoring `Retry-After`.
- **Search limits.** The empty search used for the total record count, and the sort property name (`lastmodifieddate` for contacts, `hs_lastmodifieddate` for companies and deals), should be confirmed in the current docs.
- **Old reporting data.** A property with no recent writes can still matter for reports on old records. Export before you archive.
- **Duplicates are a hint.** Similar labels can mean different events. Write down what each field represents before merging.
- **Verdicts are a ranking, not permission to delete.**