LibGuides: Government Documents: Archives of federal data and websites

Information in the Presidential Transition

About this guide

Transitions from one presidential administration to another usually involve some changes to federal government websites. The transition between the Biden Presidency and the second Trump Presidency has seen an unprecedently large number of changes to federal communications and remains a rapidly changing landscape. Some federally-produced datasets have vanished permanently or temporarily, some have reappeared altered, and the web presence of entire departments has vanished.

The information in this guide has been gathered from the organizations performing the work of archiving government websites, rescuing vulnerable data, research guides created by other libraries (especially American University), and public websites.

Table of contents

This guide is organized into several sections. Click the hyperlinks to jump to them:

Federal website changes
Archives of federal websites
Federal dataset changes
Leatherby Libraries databases containing federal data
Archives of federal data
Other transition news
Other library guides

Federal website changes

What to do if you think a government website has been redacted:

If you suspect that a government webpage has been edited or partially redacted (rather than fully removed):

See if the webpage is available in GovDiff and if there's an obvious visual change.
Compare archived versions of the same page using using the Internet Archive's GovWayback (or standard Wayback Machine) for differences between older snapshots and more recent ones.
Check the End of Term Archive. The End of Term (EOT) Archive is a collaborative project that systematically saves U.S. government websites during the transition between administrations. Since it focuses on capturing a broad swath of federal web content at key points, it might have a version that predates any redactions.

Archives of Federal websites

GovDiff Compare government websites before and after Trump’s inauguration side-by-side.
Internet Archive - Wayback Machine The Wayback Machine is the part of the Internet Archive that saves "snapshots" of webpages at different points in time. While a crawler automatically updates some sites, it also has browser extensions and mobile apps for major platforms for users to save snapshots as well.
Internet Archive - Gov Wayback GovWayback is a simple tool to quickly access archived versions of government websites from before January 20, 2025 - just add "wayback.com" after ".gov" in any government URL. GovWayback automatically redirects you to that page's archived version from the Internet Archive.
End of Term Web Archive The End of Term Web Archive captures and saves U.S. Government websites at the end of presidential administrations. The EOT has thus far preserved websites from administration changes in 2008, 2012, 2016, and 2020. We are currently accepting URL nominations for the End of Term 2024 Web Archive.
UCSD - Library GovSpeak Errors guide This guide tracks pages linked in the GovSpeak acronym list on a monthly basis. Examples of items to be included here are entity name changes, technical access problems, and apparent deactivation/discontinuance of an entity (designated as "unable to verify"); simple URL changes are not recorded here. Updated more frequently during the 2025 transition.
SJSU Library - Government Information: Eliminated, Suspended, Etc. Tracks the Trump's Administration of government U.S. federal information.
Joseph R. Biden Jr. Presidential Library Archived White House Websites The official files that make up a Presidential administration's website are preserved in the National Archives’ Executive Office of the President Electronic Records Archive. In order for the public to easily access the websites, the National Archives has taken an additional step to "freeze" the websites and make them available online.
ACAsignups.net - Links to archived versions of every CDC.gov page available as of January 27, 2025 A directory of Internet Archive links to the archived versions of all ~7,200 individual HTML pages which were available at CDC.gov as of January 27, 2025, broken into 15 pages. The links are in alphabetical order by URL hierarchy.
Abortion, Every Day - CDC Birth Control Guidelines PDF documents in the following areas: Contraception, Family Planning, Sexual Health, Vaccines, Women & Children, Youth, LGBTQIA, Intimate Partner & Sexual Violence.
Archive.org - NIH NIEHS Climate Change and Human Health Literature Portal (CCHHL) data dump (2025-02-10) Here is gathered bibliographic information about 22,695 research items (journal articles etc.) as compiled by the National Institute of Environmental Health Sciences (NIEHS; part of the US federal government's National Institutes of Health) for its Climate Change and Human Health Literature Portal (CCHHL). The format is JSON Lines. For each record, search_result gives JSON data as provided by a search-results endpoint, while record gives key-value pairs of HTML fragments from the record page for each item. search_result and record are partly but not fully redundant with each other. Also included in this archive is the Hy script used to download and process the data.
RestoredCDC.org RestoredCDC.org is an independent project and is not affiliated with, endorsed by, or associated with the Centers for Disease Control and Prevention (CDC) or any government entity. The CDC provides information free of change at CDC.gov. Note the following: 1) Due to archival on January 6, 2025, no information on recent outbreaks is available. 2) Videos have not been restored. 3) Go to data.restoredcdc.org(folder organization on-going) to access restored data.

Federal dataset changes

What to do if you think a dataset has been removed:

Search data.gov and the agency website to make sure it hasn't been relocated or renamed.
Use the Internet Archive's Wayback Machine and similar tools to see if older versions of the website contain the data.
Check if the Data Rescue Project's tracker points to a backup of the dataset.
Explore other archives linked below and see if they contain the data you're looking for. Unfortunately, the web scraping and mass file transfers used to save websites and datasets often leads to a loss of metadata and discoverability issues.

What to do if you think a dataset has been redacted:

If it is available on a web page, compare snapshots with the Wayback Machine.
Look at older versions of the web metadata. Compare file sizes, timestamps, and inspect other metadata (like the date of last modification) to see if anything changed unexpectedly. Reductions in file size or missing metadata can be indicators that parts of the data might have been removed.
Compare any copy of the dataset you possess or can find in an archive to the current version. A reduction in the number of rows or columns may indicate data has been removed.

Understanding data.gov

Data.gov is a metadata catalog, and holds the information that describes datasets rather than the datasets themselves. Scrapers working through the site have to follow links back to the original hosting location. The catalog is extensive, but scraping it does not ensure all federal data has been preserved - data may not have the appropriate linkage to appear in the catalog, or may only be available via API/FTP/other ways that the scrapers cannot access.

Justin Gosses - What people are getting wrong about data.gov

Leatherby Libraries databases containing federal data

Sage Data This link opens in a new window Over 600 customizable datasets with 14 billion data points from licensed and public domain datasets. Contains federal data from USDA, HUD, Census, VA, OPM, EPA, NBER, IRS, courts, and more.
Roper Center Archives This link opens in a new window A collection of data from U.S. polling firms covering opinion on social issues, politics, pop culture, and more. The data bank contains over 710,000 questions, response results, charts, and data sets. Users must create a free account to use certain features. Terms of Use. Coverage: 1935 - present. Contains data from 22 federal surveys.
SimplyAnalytics This link opens in a new window An application that lets users develop interactive maps and reports using thousands of demographic, business, and marketing data variables. Coverage: 2000-present. Restricted to 10 users at a time. Contains data from the Census, FBI, and NOAA.

ICPSR (Inter-university Consortium for Political and Social Research) ICPSR is one of the oldest largest research data repositories in the world. It hosts the data archives of several federal agencies, including NACJD, NIA, NHGRI, Federal Highway Administration, and USAID, and ICPSR is working to preserve at-risk datasets directly.
more...less...
You must create an account affiliated with Chapman to download data from ICPSR, but you do not need one to search and browse. Some data in ICPSR is restricted-use. Please contact your librarian or one of Chapman's ICPSR representatives if you wish to access a restricted dataset.

Archives of federal data

Data Rescue Project - The Data Rescue Tracker The Data Rescue Tracker is a collaborative tool built to catalog existing public data rescue efforts so that we can coordinate better across initiatives. The Data Rescue Tracker aims to provide a consolidated overview of who is downloading which dataset from which government websites. Are you trying to figure out if someone has already rescued a specific dataset? Check out the “Downloads” list to see which datasets have been captured already.
Harvard Law School Library Innovation Lab - mirror of data.gov This repository contains BagIt zipfiles of every dataset present on data.gov and its metadata (in JSON). data.gov includes multiple kinds of datasets, including some that link to actual data files, such as CSV files, and some that link to HTML landing pages. The process runs a "shallow crawl" that collects only the directly linked files, so some records are missing data. The host site, Source Cooperative also has a handful of other federal datasets from independent uploaders. Using BagIt files requires some understanding of JSON and the datastructures created by the scraper.
ICPSR Data Lumos DataLumos is an ICPSR archive for valuable government data resources. ICPSR has a long commitment to safekeeping and disseminating US government and other social science data. DataLumos accepts deposits of public data resources from the community and recommendations of public data resources that ICPSR itself might add to DataLumos. The archive works like other ICPSR pages, with browsable metadata and direct file downloads. Contains select data from AHRQ, CFPB, FHFA, HUD, IMLS, Census, DOL, and more.
ICPSR Unprocessed Read-only Dropbox 3,000+ files of federal health and social science datasets that will eventually be properly documented and uploaded to ICPSR but can be viewed on Dropbox for now. Contains data from CDC, Department of Education, FBI, VA, and more. May be some duplication with Lumos already.
UCSB - Public Data UCSB-hosted scrapes from many federal agency sites. Files are usually text or csv and largely un-documented. Includes data from CDC, NIH, Department of Education, USDA, SSA, NOAA, NASA, and more.
Public Environmental Data Partners The Public Environmental Data Partners are committed to preserving and providing public access to federal environmental data. We are a volunteer coalition of several environmental, justice, and policy organizations, researchers across several universities, archivists, and students who rely on federal datasets and tools to support critical research, advocacy, policy, and litigation work. To gather insights on what data to preserve, we reached out to our networks, which consist largely of environmental justice groups and networks, state and local government climate offices, and academic researchers. We compiled a large list of federal databases and tools, and prioritized them based on their relative impact, our confidence that we could archive them, and the relative effort it would take to obtain and archive them. Select datasets and recreated web tools from FEMA, EPA, CDC.
IPUMS (Integrated Public-Use Microdata Samples) IPUMS provides census and survey data from around the world integrated across time and space. IPUMS integration and documentation makes it easy to study change, conduct comparative research, merge information across data types, and analyze individuals within family and community contexts. Data and services available free of charge. Contains most Census data products.
Esri Federal Datasets Esri Federal Datasets serves the ArcGIS and broader geospatial communities by providing access to a collection of curated, ready-to-use layers that are cached and updated from federal sources. This collection, from the National Geospatial Data Asset (NGDA) Portfolio Datasets, provides national geospatial data that supports decision-making activities and can also be used as valuable resources in maps and apps. The layers are updated regularly and support access through Esri and modern OGC standards, and support export to a variety of common formats. Chapman has an institutional subscription to ArcGIS Online and Pro.
Census Reporter Census Reporter is an independent project to make data from the American Community Survey (ACS) easier to use. They are unaffiliated with the U.S. Census Bureau.
Harvard Dataverse HELD Collection The purpose of this collection, put together by members of the Salata Strengthening Communities research cluster, is to store critical climate and law datasets accessible at various locations in one place. The datasets are downloaded and uploaded with minimal modification. The collection should be helpful to any researcher working on policy, compliance, and community issues related to the environment. Contains select datasets from DOT, DOE, EPA, NOAA, CEQ, DOI.
Harvard Dataverse CAFE Collection A sub-category of the Climate Change and Health Research Coordinating Center (CAFE) page in Harvard Dataverse containing archived Federal datasets. Contains select data from EPA, DOE, USDA, NOAA, FEMA, and more on topics of climate change and health.
The Data Liberation Project Several years of data obtained via FOIA requests. Contains datasets from BOP, TSA, Peace Corps, DOT, and more.
Healthy Regions Policy Lab at UIUC Public health related datasets saved by the Healthy Regions & Policies Lab. Contains some datasets from CDC, EPA, and more.
The Internet Archive - CDC data An archive of all CDC datasets uploaded to https://data.cdc.gov/browse before January 28th, 2025. Excludes corrupt datasets and data not publicly accessible. Size: 98.5 GB.
The Climate Mirror Project A 2017 project to archive U.S. Govt. websites and datasets related to climate, climate change, and global warming prior to the first Trump administration. Contains data from NOAA and NIFC.
Academic Torrents - us-gov Public data downloaded from .gov domains and US-govt associated websites. Available via torrent client. Contains data from NORAD, archives.gov, FDA, FBI, NOAA, ODEI. Several other Federal data sets are available in other collections.
River Styx An individual grassroots uploader's collection of datasets from CDC, NHTSA, NSRDB, NOAA, and more.

Large amounts of data have been gathered by institutions, nonprofits, and individuals that are not yet documented and available. More sources will be added and descriptions updated with the contents of particular archives at a later date. Groups like Safeguarding Research & Culture and the Data Hoarders subreddit report saving many more terabytes worth of data and cultural heritage materials.

Other transition news

Other library guides

Language and links on this guide have been taken from the Data Rescue Project's work as well as from that of other libraries. You may find additional resources, including articles on data rescue efforts, at the following guides from other universities: