Support for Hedge Fund Monitor from the Office of Financial Research

This commit is contained in:
Timothy Kassis
2026-02-23 13:32:09 -08:00
parent 86b5d1d30b
commit 5490490294
11 changed files with 1229 additions and 6 deletions

View File

@@ -0,0 +1,93 @@
# API Overview
## Base URL & Versioning
```
https://data.financialresearch.gov/hf/v1
```
The API version (`v1`) is required in the URL path. Currently only v1 is available.
## Protocol & Format
- All requests use **HTTPS**
- All responses are **JSON** (except `/categories` which returns CSV)
- No authentication, API keys, or registration required
- No documented rate limits — data updates at most once per day; avoid hammering the API
## Response Patterns
Most endpoints return one of:
- An **array of `[date, value]` pairs** for time series data
- A **JSON object keyed by mnemonic** for full series (timeseries + metadata)
- A **JSON array of objects** for search/metadata listings
### Timeseries array
```json
[
["2013-03-31", -3.0],
["2013-06-30", -2.0],
["2013-09-30", -2.05]
]
```
Null values appear as `null` in the value position.
### Full series object
```json
{
"FPF-ALLQHF_NAV_SUM": {
"timeseries": {
"aggregation": [["2013-03-31", 1143832916], ...]
},
"metadata": {
"mnemonic": "FPF-ALLQHF_NAV_SUM",
"description": {
"name": "All funds: net assets (sum dollar value)",
"description": "...",
"notes": "...",
"vintage_approach": "Current vintage, as of last update",
"vintage": "",
"subsetting": "None",
"subtype": "None"
},
"schedule": {
"observation_period": "Quarterly",
"observation_frequency": "Quarterly",
"seasonal_adjustment": "None",
"start_date": "2013-03-31",
"last_update": ""
}
}
}
}
```
## Mnemonic Format
Mnemonics are unique identifiers for each time series. Format varies by dataset:
| Dataset | Pattern | Example |
|---------|---------|---------|
| fpf | `FPF-{SCOPE}_{METRIC}_{STAT}` | `FPF-ALLQHF_NAV_SUM` |
| ficc | `FICC-{SERIES}` | `FICC-SPONSORED_REPO_VOL` |
| tff | `TFF-{SERIES}` | `TFF-DLRINDEX_NET_SPEC` |
| scoos | `SCOOS-{SERIES}` | varies |
Mnemonics are **case-insensitive** in query parameters (the API normalizes to uppercase in responses).
## Subseries (label)
Each mnemonic can have multiple subseries labeled:
- `aggregation` — the main data series (always present, default returned)
- `disclosure_edits` — version of the data with certain values masked for disclosure protection
## Installation
```bash
uv add requests pandas
```
No dedicated Python client exists — use `requests` directly.

View File

@@ -0,0 +1,150 @@
# Datasets Reference
## Overview
The HFM API provides data from four source datasets. Each dataset has a short key used in API calls.
| Key | Full Name | Source | Update Frequency |
|-----|-----------|--------|-----------------|
| `fpf` | SEC Form PF | U.S. Securities and Exchange Commission | Quarterly |
| `tff` | CFTC Traders in Financial Futures | Commodity Futures Trading Commission | Monthly |
| `scoos` | Senior Credit Officer Opinion Survey on Dealer Financing Terms | Federal Reserve Board | Quarterly |
| `ficc` | FICC Sponsored Repo Service Volumes | DTCC Fixed Income Clearing Corp | Monthly |
---
## SEC Form PF (`fpf`)
The largest and most comprehensive dataset in the HFM. Covers aggregated statistics from Qualifying Hedge Fund filings.
**Who files:** SEC-registered investment advisers with ≥$150M in private fund AUM. Large Hedge Fund Advisers (≥$1.5B in hedge fund AUM) file quarterly; others file annually.
**What is a Qualifying Hedge Fund:** Any hedge fund with net assets ≥$500M advised by a Large Hedge Fund Adviser.
**Data aggregation:** OFR aggregates, rounds, and masks data to avoid disclosure of individual filer information. Winsorization is applied to remove extreme outliers.
**Strategies tracked:**
- All Qualifying Hedge Funds (`ALLQHF`)
- Equity (`STRATEGY_EQUITY`)
- Credit (`STRATEGY_CREDIT`)
- Macro (`STRATEGY_MACRO`)
- Relative value (`STRATEGY_RELVALUE`)
- Multi-strategy (`STRATEGY_MULTI`)
- Event-driven (`STRATEGY_EVENT`)
- Fund of funds (`STRATEGY_FOF`)
- Other (`STRATEGY_OTHER`)
- Managed futures/CTA (`STRATEGY_MFCTA`)
**Mnemonic naming convention:**
```
FPF-{SCOPE}_{METRIC}_{AGGREGATION_TYPE}
```
| Scope | Meaning |
|-------|---------|
| `ALLQHF` | All Qualifying Hedge Funds |
| `STRATEGY_EQUITY` | Equity strategy funds |
| `STRATEGY_CREDIT` | Credit strategy funds |
| `STRATEGY_MACRO` | Macro strategy funds |
| etc. | |
| Metric | Meaning |
|--------|---------|
| `NAV` | Net assets value |
| `GAV` | Gross assets value |
| `GNE` | Gross notional exposure |
| `BORROWING` | Total borrowing |
| `LEVERAGERATIO` | Leverage ratio |
| `CASHRATIO` | Unencumbered cash ratio |
| `GROSSRETURN` | Quarterly gross returns |
| `NETRETURN` | Quarterly net returns |
| `COUNT` | Number of qualifying funds |
| `OPENPOSITIONS` | Open positions count |
| `CDSDOWN250BPS` | Stress test: CDS -250 bps |
| `CDSUP250BPS` | Stress test: CDS +250 bps |
| `EQUITYDOWN15PCT` | Stress test: equity -15% |
| etc. | |
| Aggregation type | Meaning |
|-----------------|---------|
| `SUM` | Sum (total dollar value) |
| `GAVWMEAN` | Gross asset-weighted average |
| `NAVWMEAN` | Net asset-weighted average |
| `P5` | 5th percentile fund |
| `P50` | Median fund |
| `P95` | 95th percentile fund |
| `PCTCHANGE` | Percent change year-over-year |
| `CHANGE` | Cumulative one-year change |
| `COUNT` | Count |
**Key series examples:**
```
FPF-ALLQHF_NAV_SUM All funds: total net assets
FPF-ALLQHF_GAV_SUM All funds: total gross assets
FPF-ALLQHF_GNE_SUM All funds: gross notional exposure
FPF-ALLQHF_LEVERAGERATIO_GAVWMEAN All funds: leverage (GAV-weighted)
FPF-ALLQHF_LEVERAGERATIO_NAVWMEAN All funds: leverage (NAV-weighted)
FPF-ALLQHF_BORROWING_SUM All funds: total borrowing
FPF-ALLQHF_CDSUP250BPS_P5 Stress test: CDS +250bps (5th pct)
FPF-ALLQHF_CDSUP250BPS_P50 Stress test: CDS +250bps (median)
FPF-ALLQHF_PARTY1_SUM Largest counterparty: total lending
FPF-STRATEGY_CREDIT_NAV_SUM Credit funds: total net assets
FPF-STRATEGY_EQUITY_LEVERAGERATIO_GAVWMEAN Equity funds: leverage
```
**Data note:** Historical data starts Q1 2013 (2013-03-31). Masked values appear as `null`.
---
## CFTC Traders in Financial Futures (`tff`)
Select statistics from the CFTC Commitments of Traders (COT) report covering financial futures.
**What is tracked:** Net positioning of leveraged funds (hedge funds and commodity trading advisors) in financial futures markets, including equity index futures, interest rate futures, currency futures, and other financial instruments.
**Update frequency:** Monthly (derived from weekly CFTC COT releases)
**Key use cases:**
- Monitoring hedge fund positioning in futures markets
- Analyzing speculative vs. commercial positioning
- Tracking changes in financial futures open interest
---
## FRB SCOOS (`scoos`)
Senior Credit Officer Opinion Survey on Dealer Financing Terms conducted by the Federal Reserve Board.
**What it measures:** Survey responses from senior credit officers at major U.S. banks on terms and conditions of their securities financing and over-the-counter derivatives transactions. Covers topics including:
- Availability and terms of credit
- Collateral requirements and haircuts
- Maximum maturity of repos
- Changes in financing terms for hedge funds
**Update frequency:** Quarterly
**Key use cases:**
- Monitoring credit tightening/easing for hedge funds
- Tracking changes in dealer financing conditions
- Understanding repo market conditions from the dealer perspective
---
## FICC Sponsored Repo (`ficc`)
Statistics from the DTCC Fixed Income Clearing Corporation (FICC) Sponsored Repo Service public data.
**What it measures:** Volumes of sponsored repo and reverse repo transactions cleared through FICC's sponsored member program.
| Mnemonic | Description |
|----------|-------------|
| `FICC-SPONSORED_REPO_VOL` | Sponsored repo: repo volume |
| `FICC-SPONSORED_REVREPO_VOL` | Sponsored repo: reverse repo volume |
**Update frequency:** Monthly
**Key use cases:**
- Monitoring growth of the sponsored repo market
- Tracking volumes of centrally cleared repo activity
- Analyzing changes in repo market structure

View File

@@ -0,0 +1,196 @@
# Combined Data & Metadata Endpoints
## 1. Full Single Series — `/series/full`
**URL:** `GET https://data.financialresearch.gov/hf/v1/series/full`
Returns both timeseries data and all metadata for one series in a single call.
### Parameters
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `mnemonic` | string | **Yes** | Series identifier |
| `start_date` | string | No | Start date `YYYY-MM-DD` |
| `end_date` | string | No | End date `YYYY-MM-DD` |
| `periodicity` | string | No | Resample frequency |
| `how` | string | No | Aggregation: `last`, `first`, `mean`, `median`, `sum` |
| `remove_nulls` | string | No | `true` to remove nulls |
| `time_format` | string | No | `date` or `ms` |
### Response
```json
{
"FPF-ALLQHF_NAV_SUM": {
"timeseries": {
"aggregation": [["2013-03-31", 1143832916], ...],
"disclosure_edits": [...]
},
"metadata": { ... }
}
}
```
### Examples
```python
import requests
import pandas as pd
BASE = "https://data.financialresearch.gov/hf/v1"
resp = requests.get(f"{BASE}/series/full", params={
"mnemonic": "FPF-ALLQHF_NAV_SUM",
"start_date": "2018-01-01"
})
result = resp.json()
mnemonic = "FPF-ALLQHF_NAV_SUM"
# Extract timeseries
ts = result[mnemonic]["timeseries"]["aggregation"]
df = pd.DataFrame(ts, columns=["date", "nav_sum"])
# Extract metadata
meta = result[mnemonic]["metadata"]
print(meta["description"]["name"])
print(meta["schedule"]["observation_frequency"])
```
---
## 2. Multiple Series Full — `/series/multifull`
**URL:** `GET https://data.financialresearch.gov/hf/v1/series/multifull`
Returns data + metadata for multiple series in one request. Response is keyed by mnemonic, same structure as `/series/full`.
### Parameters
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `mnemonics` | string | **Yes** | Comma-separated mnemonics, no spaces |
| `start_date` | string | No | Start date `YYYY-MM-DD` |
| `end_date` | string | No | End date `YYYY-MM-DD` |
| `periodicity` | string | No | Resample frequency |
| `how` | string | No | Aggregation method |
| `remove_nulls` | string | No | `true` to remove nulls |
| `time_format` | string | No | `date` or `ms` |
### Examples
```python
# Fetch multiple leverage series at once
resp = requests.get(f"{BASE}/series/multifull", params={
"mnemonics": "FPF-ALLQHF_LEVERAGERATIO_GAVWMEAN,FPF-STRATEGY_EQUITY_LEVERAGERATIO_GAVWMEAN,FPF-STRATEGY_CREDIT_LEVERAGERATIO_GAVWMEAN",
"start_date": "2015-01-01",
"remove_nulls": "true"
})
results = resp.json()
# Build a combined DataFrame
frames = []
for mne, data in results.items():
ts = data["timeseries"]["aggregation"]
df = pd.DataFrame(ts, columns=["date", mne])
df["date"] = pd.to_datetime(df["date"])
df = df.set_index("date")
frames.append(df)
combined = pd.concat(frames, axis=1)
```
---
## 3. Full Dataset — `/series/dataset`
**URL:** `GET https://data.financialresearch.gov/hf/v1/series/dataset`
Without parameters: returns basic info about all datasets.
With `dataset=`: returns all series in that dataset with full data.
> **Warning:** Dataset responses can be very large. Use `start_date` to limit the data range for performance.
### Parameters
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `dataset` | string | No | Dataset key: `fpf`, `tff`, `scoos`, `ficc` |
| `vintage` | string | No | `p` (preliminary), `f` (final), `a` (as of). Default: all |
| `start_date` | string | No | Start date `YYYY-MM-DD` |
| `end_date` | string | No | End date `YYYY-MM-DD` |
| `periodicity` | string | No | Resample frequency |
| `how` | string | No | Aggregation method |
| `remove_nulls` | string | No | `true` to remove nulls |
| `time_format` | string | No | `date` or `ms` |
### Examples
```python
# List all available datasets
resp = requests.get(f"{BASE}/series/dataset")
datasets = resp.json()
# {"ficc": {"long_name": "...", "short_name": "..."}, "fpf": {...}, ...}
# Download full FPF dataset (recent data only)
resp = requests.get(f"{BASE}/series/dataset", params={
"dataset": "fpf",
"start_date": "2020-01-01"
})
fpf_data = resp.json()
# fpf_data["short_name"], fpf_data["long_name"]
# fpf_data["timeseries"]["FPF-ALLQHF_NAV_SUM"]["timeseries"]["aggregation"]
# Annual data with custom periodicity
resp = requests.get(f"{BASE}/series/dataset", params={
"dataset": "fpf",
"start_date": "2015-01-01",
"end_date": "2024-12-31",
"periodicity": "A",
"how": "last"
})
# Only final vintage
resp = requests.get(f"{BASE}/series/dataset", params={
"dataset": "ficc",
"vintage": "f"
})
```
---
## 4. Category Data — `/categories`
**URL:** `GET https://data.financialresearch.gov/hf/v1/categories`
Returns a **CSV file** with all series data for a given category.
### Parameters
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `category` | string | **Yes** | Category key |
### Available Categories
| Key | Description |
|-----|-------------|
| `complexity` | Open positions, strategy distribution, asset class exposure |
| `counterparties` | Counterparty concentration and prime broker lending |
| `leverage` | Leverage ratios, borrowing, gross notional exposure |
| `liquidity` | Financing maturity, investor redemption terms, portfolio liquidity |
| `risk_management` | Stress test results |
| `size` | Industry size (AUM, fund count, net/gross assets) |
### Examples
```python
# Download leverage category as CSV
resp = requests.get(f"{BASE}/categories", params={"category": "leverage"})
# Response is CSV text
import io
df = pd.read_csv(io.StringIO(resp.text))
# Also accessible via direct URL:
# https://data.financialresearch.gov/hf/v1/categories?category=leverage
```

View File

@@ -0,0 +1,136 @@
# Metadata Endpoints
## 1. List Mnemonics — `/metadata/mnemonics`
**URL:** `GET https://data.financialresearch.gov/hf/v1/metadata/mnemonics`
Returns all series identifiers available through the API.
### Parameters
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `dataset` | string | No | Filter by dataset key: `fpf`, `tff`, `scoos`, `ficc` |
| `output` | string | No | `by_dataset` — returns a hash grouped by dataset |
### Examples
```python
import requests
BASE = "https://data.financialresearch.gov/hf/v1"
# All mnemonics (flat list)
resp = requests.get(f"{BASE}/metadata/mnemonics")
mnemonics = resp.json()
# Returns: ["FPF-ALLQHF_CDSDOWN250BPS_P5", "FPF-ALLQHF_CDSDOWN250BPS_P50", ...]
# Mnemonics for a single dataset with names
resp = requests.get(f"{BASE}/metadata/mnemonics", params={"dataset": "fpf"})
# Returns: [{"mnemonic": "FPF-ALLQHF_CDSDOWN250BPS_P5", "series_name": "Stress test: CDS spreads decrease 250 basis points net impact on NAV (5th percentile fund)"}, ...]
# All mnemonics grouped by dataset
resp = requests.get(f"{BASE}/metadata/mnemonics", params={"output": "by_dataset"})
grouped = resp.json()
# Returns: {"ficc": [{mnemonic, series_name}, ...], "fpf": [...], ...}
```
---
## 2. Single Series Query — `/metadata/query`
**URL:** `GET https://data.financialresearch.gov/hf/v1/metadata/query`
Returns full metadata for a single mnemonic.
### Parameters
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `mnemonic` | string | **Yes** | The series mnemonic |
| `fields` | string | No | Comma-separated list of fields to retrieve. Use `/` to access subfields (e.g., `release/long_name`) |
### Metadata Fields
The metadata object includes these top-level fields (with subfields):
| Field | Subfields |
|-------|-----------|
| `mnemonic` | — |
| `description` | `name`, `description`, `notes`, `vintage_approach`, `vintage`, `subsetting`, `subtype` |
| `schedule` | `observation_period`, `observation_frequency`, `seasonal_adjustment`, `start_date`, `last_update` |
| `release` | `long_name`, `short_name`, and other release-level metadata |
### Examples
```python
# Full metadata
resp = requests.get(f"{BASE}/metadata/query", params={
"mnemonic": "fpf-allqhf_cdsup250bps_p5"
})
meta = resp.json()
print(meta["description"]["name"])
print(meta["schedule"]["start_date"])
print(meta["schedule"]["observation_frequency"])
# Specific subfield only
resp = requests.get(f"{BASE}/metadata/query", params={
"mnemonic": "fpf-allqhf_cdsup250bps_p5",
"fields": "release/long_name"
})
# Returns: {"release": {"long_name": "Hedge Fund Aggregated Statistics from SEC Form PF Filings"}}
# Multiple fields
resp = requests.get(f"{BASE}/metadata/query", params={
"mnemonic": "fpf-allqhf_cdsup250bps_p5",
"fields": "description/name,schedule/start_date,schedule/observation_frequency"
})
```
---
## 3. Series Search — `/metadata/search`
**URL:** `GET https://data.financialresearch.gov/hf/v1/metadata/search`
Full-text search across all metadata fields. Supports wildcards.
### Parameters
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `query` | string | **Yes** | Search string. Supports `*` (multi-char wildcard) and `?` (single-char wildcard) |
### Response Fields
Each result object contains:
| Field | Description |
|-------|-------------|
| `mnemonic` | Series identifier (or `"none"` for dataset-level metadata) |
| `dataset` | Dataset key (`fpf`, `tff`, `scoos`, `ficc`) |
| `field` | Which metadata field matched (e.g., `description/name`) |
| `value` | The matched field value |
| `type` | Data type (`str`, etc.) |
### Examples
```python
# Find series containing "leverage" anywhere
resp = requests.get(f"{BASE}/metadata/search", params={"query": "*leverage*"})
results = resp.json()
for r in results:
print(r["mnemonic"], r["field"], r["value"])
# Find series starting with "Fund"
resp = requests.get(f"{BASE}/metadata/search", params={"query": "Fund*"})
# Find by exact dataset name
resp = requests.get(f"{BASE}/metadata/search", params={"query": "FICC*"})
# Search for stress test series
resp = requests.get(f"{BASE}/metadata/search", params={"query": "*stress*"})
# Get unique mnemonics from search results
results = resp.json()
mnemonics = list({r["mnemonic"] for r in results if r["mnemonic"] != "none"})
```

View File

@@ -0,0 +1,126 @@
# Series Data Endpoints
## 1. Single Timeseries — `/series/timeseries`
**URL:** `GET https://data.financialresearch.gov/hf/v1/series/timeseries`
Returns date/value pairs for a single series.
### Parameters
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `mnemonic` | string | **Yes** | Series identifier |
| `label` | string | No | Subseries: `aggregation` (default) or `disclosure_edits` |
| `start_date` | string | No | First date `YYYY-MM-DD` (default: `1901-01-01`) |
| `end_date` | string | No | Last date `YYYY-MM-DD` (default: today) |
| `periodicity` | string | No | Resample to frequency (see parameters.md) |
| `how` | string | No | Aggregation method: `last` (default), `first`, `mean`, `median`, `sum` |
| `remove_nulls` | string | No | `true` to remove null values |
| `time_format` | string | No | `date` (YYYY-MM-DD, default) or `ms` (epoch milliseconds) |
### Response
Array of `[date_string, value]` pairs. Values are floats or `null`.
```json
[
["2013-03-31", -3.0],
["2013-06-30", -2.0],
["2013-09-30", null],
["2013-12-31", -3.0]
]
```
### Examples
```python
import requests
import pandas as pd
BASE = "https://data.financialresearch.gov/hf/v1"
# Full history for a series
resp = requests.get(f"{BASE}/series/timeseries", params={
"mnemonic": "FPF-ALLQHF_LEVERAGERATIO_GAVWMEAN"
})
data = resp.json()
df = pd.DataFrame(data, columns=["date", "leverage"])
df["date"] = pd.to_datetime(df["date"])
# Filtered date range with null removal
resp = requests.get(f"{BASE}/series/timeseries", params={
"mnemonic": "FPF-ALLQHF_NAV_SUM",
"start_date": "2018-01-01",
"end_date": "2024-12-31",
"remove_nulls": "true"
})
# Annual frequency (calendar year end)
resp = requests.get(f"{BASE}/series/timeseries", params={
"mnemonic": "FPF-ALLQHF_GAV_SUM",
"periodicity": "A",
"how": "last"
})
# Epoch milliseconds for charting libraries
resp = requests.get(f"{BASE}/series/timeseries", params={
"mnemonic": "FICC-SPONSORED_REPO_VOL",
"time_format": "ms"
})
```
---
## 2. Series Spread — `/calc/spread`
**URL:** `GET https://data.financialresearch.gov/hf/v1/calc/spread`
Returns the difference (spread) between two series: `x - y`. Useful for comparing rates or examining basis relationships.
### Parameters
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `x` | string | **Yes** | Base series mnemonic |
| `y` | string | **Yes** | Subtracted series mnemonic |
| `start_date` | string | No | Start date `YYYY-MM-DD` |
| `end_date` | string | No | End date `YYYY-MM-DD` |
| `periodicity` | string | No | Resample frequency |
| `how` | string | No | Aggregation: `last`, `first`, `mean`, `median`, `sum` |
| `remove_nulls` | string | No | `true` to remove nulls |
| `time_format` | string | No | `date` or `ms` |
### Response
Array of `[date, value]` pairs where value = x - y at each date.
```json
[
["2020-01-02", 0.15],
["2020-03-03", -0.37],
["2020-04-01", 0.60]
]
```
### Examples
```python
# Spread between two repo rates
resp = requests.get(f"{BASE}/calc/spread", params={
"x": "REPO-GCF_AR_G30-P",
"y": "REPO-TRI_AR_AG-P",
"start_date": "2019-01-01",
"remove_nulls": "true"
})
spread = pd.DataFrame(resp.json(), columns=["date", "spread_bps"])
spread["date"] = pd.to_datetime(spread["date"])
# Annual spread with mean aggregation
resp = requests.get(f"{BASE}/calc/spread", params={
"x": "FPF-STRATEGY_EQUITY_LEVERAGERATIO_GAVWMEAN",
"y": "FPF-STRATEGY_CREDIT_LEVERAGERATIO_GAVWMEAN",
"periodicity": "A",
"how": "mean"
})
```

View File

@@ -0,0 +1,287 @@
# Code Examples
## Installation
```bash
uv add requests pandas matplotlib
```
## 1. Discover Available Data
```python
import requests
import pandas as pd
BASE = "https://data.financialresearch.gov/hf/v1"
# List all datasets
resp = requests.get(f"{BASE}/series/dataset")
for key, info in resp.json().items():
print(f"{key}: {info['long_name']}")
# List all mnemonics for FPF with names
resp = requests.get(f"{BASE}/metadata/mnemonics", params={"dataset": "fpf"})
mnemonics = pd.DataFrame(resp.json())
print(mnemonics.head(20))
# Search for leverage-related series
resp = requests.get(f"{BASE}/metadata/search", params={"query": "*leverage*"})
results = pd.DataFrame(resp.json())
# Deduplicate to get unique mnemonics
leverage_series = results[results["mnemonic"] != "none"]["mnemonic"].unique()
print(leverage_series)
```
## 2. Fetch and Plot Hedge Fund Leverage Over Time
```python
import requests
import pandas as pd
import matplotlib.pyplot as plt
BASE = "https://data.financialresearch.gov/hf/v1"
# Fetch overall leverage ratio
resp = requests.get(f"{BASE}/series/timeseries", params={
"mnemonic": "FPF-ALLQHF_LEVERAGERATIO_GAVWMEAN",
"remove_nulls": "true"
})
df = pd.DataFrame(resp.json(), columns=["date", "leverage"])
df["date"] = pd.to_datetime(df["date"])
# Get metadata
meta_resp = requests.get(f"{BASE}/metadata/query", params={
"mnemonic": "FPF-ALLQHF_LEVERAGERATIO_GAVWMEAN",
"fields": "description/name,schedule/observation_frequency"
})
meta = meta_resp.json()
title = meta["description"]["name"]
plt.figure(figsize=(12, 5))
plt.plot(df["date"], df["leverage"], linewidth=2)
plt.title(title)
plt.ylabel("Leverage Ratio")
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig("hedge_fund_leverage.png", dpi=150)
```
## 3. Compare Strategy-Level Leverage
```python
import requests
import pandas as pd
import matplotlib.pyplot as plt
BASE = "https://data.financialresearch.gov/hf/v1"
strategies = {
"All Funds": "FPF-ALLQHF_LEVERAGERATIO_GAVWMEAN",
"Equity": "FPF-STRATEGY_EQUITY_LEVERAGERATIO_GAVWMEAN",
"Credit": "FPF-STRATEGY_CREDIT_LEVERAGERATIO_GAVWMEAN",
"Macro": "FPF-STRATEGY_MACRO_LEVERAGERATIO_GAVWMEAN",
}
resp = requests.get(f"{BASE}/series/multifull", params={
"mnemonics": ",".join(strategies.values()),
"remove_nulls": "true"
})
results = resp.json()
fig, ax = plt.subplots(figsize=(14, 6))
for label, mne in strategies.items():
ts = results[mne]["timeseries"]["aggregation"]
df = pd.DataFrame(ts, columns=["date", "value"])
df["date"] = pd.to_datetime(df["date"])
ax.plot(df["date"], df["value"], label=label, linewidth=2)
ax.set_title("Hedge Fund Leverage by Strategy (GAV-Weighted)")
ax.set_ylabel("Leverage Ratio")
ax.legend()
ax.grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig("leverage_by_strategy.png", dpi=150)
```
## 4. Download Full FPF Dataset into a Wide DataFrame
```python
import requests
import pandas as pd
BASE = "https://data.financialresearch.gov/hf/v1"
# Download entire FPF dataset, recent data only
resp = requests.get(f"{BASE}/series/dataset", params={
"dataset": "fpf",
"start_date": "2015-01-01",
"remove_nulls": "false"
})
data = resp.json()
# Build a wide DataFrame with one column per series
frames = {}
for mne, series_data in data["timeseries"].items():
ts = series_data["timeseries"]["aggregation"]
if ts:
s = pd.Series(
{row[0]: row[1] for row in ts},
name=mne
)
frames[mne] = s
df = pd.DataFrame(frames)
df.index = pd.to_datetime(df.index)
df = df.sort_index()
print(f"Shape: {df.shape}") # (dates, series)
print(df.tail())
```
## 5. Stress Test Analysis
```python
import requests
import pandas as pd
BASE = "https://data.financialresearch.gov/hf/v1"
# CDS stress test scenarios (P5 = 5th percentile fund, P50 = median fund)
stress_mnemonics = [
"FPF-ALLQHF_CDSDOWN250BPS_P5",
"FPF-ALLQHF_CDSDOWN250BPS_P50",
"FPF-ALLQHF_CDSUP250BPS_P5",
"FPF-ALLQHF_CDSUP250BPS_P50",
]
resp = requests.get(f"{BASE}/series/multifull", params={
"mnemonics": ",".join(stress_mnemonics),
"remove_nulls": "true"
})
results = resp.json()
frames = []
for mne in stress_mnemonics:
ts = results[mne]["timeseries"]["aggregation"]
name = results[mne]["metadata"]["description"]["name"]
df = pd.DataFrame(ts, columns=["date", mne])
df["date"] = pd.to_datetime(df["date"])
df = df.set_index("date")
frames.append(df)
stress_df = pd.concat(frames, axis=1)
stress_df.columns = [r["metadata"]["description"]["name"]
for r in [results[m] for m in stress_mnemonics]]
print(stress_df.tail(8).to_string())
```
## 6. FICC Sponsored Repo Volume Trend
```python
import requests
import pandas as pd
import matplotlib.pyplot as plt
BASE = "https://data.financialresearch.gov/hf/v1"
resp = requests.get(f"{BASE}/series/multifull", params={
"mnemonics": "FICC-SPONSORED_REPO_VOL,FICC-SPONSORED_REVREPO_VOL",
"remove_nulls": "true"
})
results = resp.json()
fig, ax = plt.subplots(figsize=(12, 5))
for mne, label in [
("FICC-SPONSORED_REPO_VOL", "Repo Volume"),
("FICC-SPONSORED_REVREPO_VOL", "Reverse Repo Volume"),
]:
ts = results[mne]["timeseries"]["aggregation"]
df = pd.DataFrame(ts, columns=["date", "value"])
df["date"] = pd.to_datetime(df["date"])
# Convert to trillions
df["value"] = df["value"] / 1e12
ax.plot(df["date"], df["value"], label=label, linewidth=2)
ax.set_title("FICC Sponsored Repo Service Volumes")
ax.set_ylabel("Trillions USD")
ax.legend()
ax.grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig("ficc_repo_volumes.png", dpi=150)
```
## 7. Download Category CSV
```python
import requests
import io
import pandas as pd
BASE = "https://data.financialresearch.gov/hf/v1"
# Download the leverage category as a DataFrame
resp = requests.get(f"{BASE}/categories", params={"category": "leverage"})
df = pd.read_csv(io.StringIO(resp.text))
print(df.head())
# All categories: complexity, counterparties, leverage, liquidity, risk_management, size
```
## 8. Counterparty Concentration Analysis
```python
import requests
import pandas as pd
BASE = "https://data.financialresearch.gov/hf/v1"
# Top 8 counterparties lending to all qualifying hedge funds
party_mnemonics = [f"FPF-ALLQHF_PARTY{i}_SUM" for i in range(1, 9)]
resp = requests.get(f"{BASE}/series/multifull", params={
"mnemonics": ",".join(party_mnemonics),
"remove_nulls": "false"
})
results = resp.json()
# Get the most recent quarter's values
frames = []
for mne in party_mnemonics:
ts = results[mne]["timeseries"]["aggregation"]
df = pd.DataFrame(ts, columns=["date", "value"])
df["date"] = pd.to_datetime(df["date"])
df["mnemonic"] = mne
frames.append(df)
all_data = pd.concat(frames).pivot(index="date", columns="mnemonic", values="value")
print("Most recent quarter counterparty exposure (USD billions):")
print((all_data.iloc[-1] / 1e9).sort_values(ascending=False).to_string())
```
## 9. Periodic Refresh Pattern
```python
import requests
import pandas as pd
from datetime import datetime, timedelta
BASE = "https://data.financialresearch.gov/hf/v1"
def get_recent_fpf(days_back: int = 180) -> pd.DataFrame:
"""Fetch only the most recent FPF observations (for periodic refreshes)."""
start = (datetime.today() - timedelta(days=days_back)).strftime("%Y-%m-%d")
resp = requests.get(f"{BASE}/series/dataset", params={
"dataset": "fpf",
"start_date": start,
"remove_nulls": "true"
})
data = resp.json()
frames = {}
for mne, series_data in data["timeseries"].items():
ts = series_data["timeseries"]["aggregation"]
if ts:
frames[mne] = pd.Series({row[0]: row[1] for row in ts}, name=mne)
return pd.DataFrame(frames)
recent = get_recent_fpf(days_back=365)
print(recent.shape)
```

View File

@@ -0,0 +1,104 @@
# Parameters Reference
## Periodicity Codes
Used in `periodicity` parameter for `/series/timeseries`, `/series/full`, `/series/multifull`, `/series/dataset`, and `/calc/spread`.
| Code | Description |
|------|-------------|
| `A` | Calendar Year End |
| `AS` | Calendar Year Start |
| `D` | Daily |
| `M` | Calendar Month End |
| `MS` | Calendar Month Start |
| `W` | Weekly (Sunday Start) |
| `B` | Business Day (Weekday) |
| `BM` | Business Month End |
| `BMS` | Business Month Start |
| `Q` | Quarter End |
| `BQ` | Business Quarter End |
| `QS` | Quarter Start |
| `BQS` | Business Quarter Start |
| `BA` | Business Year End |
| `BAS` | Business Year Start |
**Note:** When resampling, the `how` parameter specifies how to compute the value within each period.
## Aggregation Methods (`how`)
| Value | Description |
|-------|-------------|
| `last` | Last value of the period (default) |
| `first` | First value of the period |
| `mean` | Mean (average) of all values in the period |
| `median` | Median of all values in the period |
| `sum` | Sum of all values in the period |
## Vintage (`vintage` — dataset endpoint only)
| Value | Description |
|-------|-------------|
| `p` | Preliminary data |
| `f` | Final data |
| `a` | "As of" data |
If not specified, all vintages (preliminary, final, and "as of") are returned together.
## Date Parameters
- `start_date` and `end_date` use `YYYY-MM-DD` format
- Default `start_date`: `1901-01-01` (all available history)
- Default `end_date`: today's date (all available up to now)
- FPF data starts from `2013-03-31`; FICC/TFF data start dates vary by series
## Time Format (`time_format`)
| Value | Format |
|-------|--------|
| `date` | String in `YYYY-MM-DD` format (default) |
| `ms` | Integer: milliseconds since Unix epoch (1970-01-01) |
The `ms` format is useful for JavaScript charting libraries (e.g., Highcharts, D3).
## Label (`label` — timeseries endpoint only)
| Value | Description |
|-------|-------------|
| `aggregation` | Main aggregated series (default) |
| `disclosure_edits` | Series with disclosure-masked values |
## Null Handling
- `remove_nulls=true` — removes all `[date, null]` pairs from the response
- Without this parameter, nulls are included as `null` in the value position
- FPF masked values (withheld for disclosure protection) appear as `null`
## Search Wildcards
Used in the `query` parameter of `/metadata/search`:
| Wildcard | Matches |
|----------|---------|
| `*` | Zero or more characters |
| `?` | Exactly one character |
Examples:
- `Fund*` — anything starting with "Fund"
- `*credit*` — anything containing "credit"
- `FPF-ALLQHF_?` — mnemonics starting with `FPF-ALLQHF_` followed by one char
## Field Selectors
Used in `fields` parameter of `/metadata/query`. Access subfields with `/`:
```
fields=description/name
fields=schedule/start_date,schedule/observation_frequency
fields=release/long_name,description/description
```
Available top-level fields:
- `mnemonic`
- `description` (subfields: `name`, `description`, `notes`, `vintage_approach`, `vintage`, `subsetting`, `subtype`)
- `schedule` (subfields: `observation_period`, `observation_frequency`, `seasonal_adjustment`, `start_date`, `last_update`)
- `release` (subfields: `long_name`, `short_name`, and others depending on the series)