8 Data inputs
8.1 Pricing
Final data: Pricing for beef along the supply chain .
8.1.1 Differentiated
ADD
8.1.2 Commodity
Here we focus on price data for the last 10 years, where available. All prices are reported in 2024 dollars.
8.1.2.1 Monthly prices along supply chain
8.1.2.1.1 PPI
For prices at the farmgate, we use the producer price index by commodity: Farm products: Slaughter Cattle(“WPU0131.csv”) to put all beef price data in 2024 dollars.
Citation: U.S. Bureau of Labor Statistics, Producer Price Index by Commodity: Farm Products: Slaughter Cattle [WPU0131], retrieved from FRED, Federal Reserve Bank of St. Louis; https://fred.stlouisfed.org/series/WPU0131, August 17, 2026.
For retaila and wholesale prices, we use the Producer Price Index by Commodity: Processed Foods and Feeds: Meats (WPU0221)(“WPU0221.csv”).
Citation: U.S. Bureau of Labor Statistics, Producer Price Index by Commodity: Processed Foods and Feeds: Meats [WPU0221], retrieved from FRED, Federal Reserve Bank of St. Louis; https://fred.stlouisfed.org/series/WPU0221, August 17, 2026.
We compute an average yearly PPI and then use that to convert all dollars into 2024 dollars. We keep data starting in 2014.
8.1.2.1.2 Figures
8.1.2.2 Slaughter cattle
We pull data from USDA AMS My Market News: Commodities: Slaughter Cattle, Report: New York Weekly Auction Summary, Report Section: Report Details.
Pull historical USDA MyMarketNews (MARS) data Report: New York Weekly Cattle Auction Summary (Slug ID 2011 / AMS_2011) Commodity: Slaughter Cattle Range: 2019-05-03 to present
Note that you will have to use your own API key.
8.1.2.2.1 Figures
8.1.3 Organic
Using data from the USDA AMS Livestock, Poultry, and Grain Market News, New York Dept of Ag Mrkt News, Feb 5, 2025, we estimate the increased cost of organic beef at the farmgate.
Data are pulled from Slaughter Cattle, Dairy Cows, we use DAIRY COWS - Lean 85-90% (Per Cwt / Actual Wt) as it has the largest sample size. The average price per cwt/actual wt for conventional is 116.98 and for organic is 118.78. Organic is 1.5% higher than conventional. We apply a 1.8% premium on farmgate organic prices.
Retail prices are found from the USDA AMS Livestock, Poultry, & Grain Market News, Weekly Grocery Store Beef Feature Activity, August 14, 2026. We apply the prices differences of conventional to organic to both retail and wholesale. We use the price of ground beef, Ground Beef 80-89%, 1-2 Lbs from the Previous Week (PW).
Conventional: 6.26 USDA Organic, fresh: 6.99
We apply a 11.7% premium on all retail and wholesale organic prices.
8.2 Culling decisions
We need to characterize the relationship between cull rate and the relationship between milk and beef prices. Monthly price data for farmgate milk and beef prices.
Milk prices received (dollars per cwt) are from USDA NASS. Prices are reported in 2024 dollars using the Producer Price Index by Commodity: Farm Products: Raw Milk (WPS016).
8.2.0.1 Milk
8.2.0.2 Cull beef
Change once I get a better source of data. For now, I’m using the data provided above.
8.2.0.3 Save
8.2.1 Final table
For the model, we include one worksheet with differentiated prices, one with commodity beef prices at each level of the supply chain. These prices are an average of the prices from 2022-2024. All prices are in 2024 dollars.
Convert all prices into $/kg. 1 kg = 2.20462 lb
cents/kg=cents/lb×2.20462
$/kg=100cents/lb×2.20462
We also keep a table with all of the data to be used for validation.
8.3 Agricultural production
8.3.1 Farmer characteristics
We use data from the restricted access 2022 Census of Agriculture to paramertize agents in the model. Exported data and code used to generate results.
The sample of farms include farms in New York with primary commodity dairy and at least $1,000 in sales. Another way to disaggreate data by scale is to use herd size, where operations with less than 300 dairy cows are small and 300 or more are large.
- For our sample of farms with gross cash farm income (GCFI) <$1M, the average dairy herd size is around 60.
- The correlation between the continuous variables GCFI and dairy herd size is 0.96.
- Almost all dairies with 300+ cows are on operations with GCFI $1M plus. – Of those farms with 10-300 cows, 15% are on farms with GCFI $M plus and 75% are on farms with GCFI<$1M.
Due to the high correlation between GCFI under/over $1M and herd size under/over 300, we use GCFI to indicate scale.
The sheet titled “summary” includes summary statistics (mean/standard error) for a variety of different farm-level characteristics by scale and MWBE.
The sheet titled “metadata” contains definitions of all variables and the sheet titled “info” describes the data at a high level along with the sample.
Notes: Operations with less than 1,000 in sales are dropped, if N<5 it is not included, (D) indicated not included due to disclosure risk.
8.4 County data
We download 2022 data from U.S. Department of Agriculture, National Agricultural Statistics Service (USDA NASS), Data, Large Datasets.
We include data on the number of farms with at least 10 milk cows and milk cow inventory by county and total for the state. The restricted access data includes New York farms with primary commodity dairy and at least 1,000 in sales, N=2,163. This is very similar to the definition of a commercial dairy farm of 10 or more dairy cows (which we use in this section) where N=2,165.
However, milk cow inventory on farms with less than 10 dairy cows is not disclosed in some counties. All farms with inventory are counted, non-disclosed data is only for inventory of milk cows. Note this this is a slightly different definition that used in the restricted access data, but the two sample sizes are very closely alighned (no )
There are a total of 2,931 dairy cows not allocated across 7 counties due to disclosure. For commercial dairy farms (at least 10 dairy cows), this is 2,317. The number of commercial dairy farms in these 7 counties are mostly 0 or 1 (1 county has 2 and another has 5). This is less than 1% of milk cow inventory in the state.
We take the 2,317 milk cow inventory that was not disclosed and allocate it evenly across the counties with missing data based evenly split based on the number of commercial dairy farms.
8.4.0.1 Allocate missing
8.4.0.2 Save
8.5 Processing and distribution
8.5.1 Description of Data Sources
All of the data from this file will be used to inform the supply chain of the ABM model.
Processors We include all USDA inspected meat processing facilities
The following information is needed for all businesses:
- Location (in NY or not)
- Address
- Role in the supply chain
- Has the business sold to New York City
- If the business sold to NYC, was the product from NY
- MWBE or not
- Works in the local and regional food supply chain
Auctions We define a list of livestock auctions in New York from
Wholesalers and Distributors We define the list of wholesale/distributor businesses from the SimplyAnalytics database of businesses located in New York State (NYS) that could carry beans. We assume that any wholesaler/distributor can carry MWBE identified product or local identified product. We also assume that wholesalers in NYS can supply all product demanded by New York City (NYC).
The only businesses that are explicity included in the model are those that are identified as MWBE businesses or those that sold to NYC previously.
Location All data sets have information on if the business is located in NYS or not
Address We include an address for each business and then convert this to latitude and longitude coordinates to be included in the map. This data is not available for the majority of businesses so addresses are gathered and input manually.
Role in the supply chain For those businesses that are wholesalers/distributors, we have that information in all data sets.
Has the business sold to NYC Any business that is in the NYC data we indicate has sold to NYC, we assume all other businesses have not.
MWBE These data are listed in the NYC data and the interview data. For the SimplyAnalytics data, in order to determine if businesses are MWBE, we use a New York State database and a NYC database.
Works with local farms Businesses that sell locally are assumed to be those that were interviewed as part of this project and those found in the Vendor Contact List in the NY Food Product Database.
Max inventory This represents the maximum inventory that can flow through the businesses. If there is no supply contraints then we use the number of pounds of beans that NYC purchased as the max inventory with the goal that this number makes it so there is no constraint on how much the business can purchase in the model.
Organic We assume that all wholesalers/distributors can carry organic producers. Organic processors are identified in FSIS data if organic is in their name, there is no formal identifier in the data.
8.5.2 Meat, Poultry and Egg product inspection directory
We pull publicly available data from the U.S. Department of Agriculture, Food Safety and Inspection Service USDA FSIS), Meat, Poultry and Egg Product Inspection Directory. The data is updated weekly and downloaded on 5/15/2026.
We keep only those establishments that listed activities livestock slaughter or livestock processing.
The Meat, Poultry and Egg Product Inspection (MPI) Directory is a listing of establishments that produce meat, poultry, and/or egg products regulated by FSIS. The Establishment Demographic Data includes additional establishment information about FSIS regulated establishments, including size, species slaughtered and aggregate categorical production information. These data are updated weekly, and the current edition replaces all previous editions.
Size The size of the establishment based on the business class. The options are Large, Small, and Very Small. Note: This variable does not correlate highly with production volume.
- Large establishments are defined as establishments with 500 or more employees
- Small establishments are defined as establishments with 10 or more employees but fewer than 500 employees
- Very Small establishments are defined as establishments with fewer than 10 employees or annual sales of less than $2.5 million
8.5.2.1 Import and clean
8.5.2.2 Define variables
8.5.3 Auctions
We create a list of livestock auctions from three sources:
- AllHay.com
- New York Agriculture and Markets
- USDA Agricultural Marketing Service - Packers and Stockyards Division
Data is imported from each source and aggregated into one file.
8.5.4 SimplyAnalytics
This data is used for the wholesale businesses.
Simply Analytics is a web-based mapping application that allows users to map bushiness by name or by industry. Data are available to Cornell researchers through the Management Library. We collected data from (ADD YEAR) on the NAICS codes that are relevant to processing, distribution and wholesaling of dry beans, beef and leafy greens.
We use data aggregated at the 6-digit NAICS code.
The NAICS codes of interest for us include:
- 424420 Packaged Frozen Food Merchant Wholesalers
- 424410 General Line Grocery Merchant Wholesalers
- 424420 Packaged Frozen Food Merchant Wholesalers
- 424470 Meat and Meat Product Merchant Wholesalers
- 424490 Other Grocery and Related Products Merchant Wholesalers
- 484110 General Freight Trucking, Local
- 484121 General Freight Trucking, Long-Distance, Truckload
- 484122 General Freight Trucking, Long-Distance, Less Than Truckload
- 493110 General Warehousing and Storage
- 493120 Refrigerated Warehousing and Storage
- 493130 Farm Product Warehousing and Storage
- 493190 Other Warehousing and Storage
8.5.4.1 Import and clean data
8.5.4.2 Define variables
Supply chain role For the the processing sectors of the supply chain, we specify the commodity, and for the wholesale/distribution sectors we assume the sell all three commodities.
Located in NY or not We create a dummy variable with a 1 if in NY and 0 otherwise. All of these businesses are located in NYS.
8.5.5 New York City Purchasing
Overall, we use the NYC purchasing data to get a list of processor businesses that are located in NYS and have sold to NYC or are located outside of NYS but sell NYS source identified products to NYC.
We also include a list of wholesalers that we will match with the Simply Analytics database to add additional information to the existing Simply Analytics data.
Here we want the data from Citywide, which is what is downloaded when you download the full purchasing data from the NYC Food Policy Dashboard.
We keep all data relevant for the year 2023, based on primary food product category equal to beef. In these data we have company names but we do not have information on their location.
The column called ny_state_spend_y_n indicates if an item is grown, processed, manufactured, or distributed by businesses located within New York State. The challenge is that the data includes up to three companies per line (origin_detail, distributor, and vendor) and for all three of these only one value for ny_state_spend_y_n. Therefore at least one, but not necessarily all of these businesses are in NYS. We want to identify businesses that sold NYS identified products to NYC, but using these data, we can’t know which of the three businesses in each row was the one identified. As a first step, we assume all businesses in the row (i.e., origin_detail, distributor, and vendor) are assumed to supply NYS product.
We assume all businesses listed under origin_detail are processors and get the beef based on the products they sold and all other businesses are wholesaler/distributors.
We want to include all businesses from this data set that are located in NYS and all out-of-state businesses that purchased NY identified products and sold to NYC (i.e., ny_state_spend_y_n==“y”), indicated by the variable “nys_product_purchase” equal to 1 if purchased source identified NY product and 0 otherwise.
While we do have a column in this dataset that indicates MWBE, we do not know from this data which business along the supply chain is MWBE. But since we have a list of all certified MWBE businesses for NYC and NYS, we rely on that list to confirm if a business is MWBE or not (rather than using the NYC purchasing database).
We want to match data for all businesses listed. For each row there is a column called origin detail, distributor and vendor. We create a new column called company_name that includes all of the businesses listed in each column. We do not retain the columns pertaining to how much was purchased by NYC.
8.5.5.1 Import and clean
8.5.5.2 Join NYC with SimplyAnalytics
Here we take the NYC wholesale data information and join it with the Simply Analytics data.
8.5.5.2.1 NYC data is the base
We add addresses from the SA data to wholesalers that sold to NYC. And then we manually add in any missing addresses. This is one set of wholesale businesses that will be included in the model.
8.5.5.2.2 Simply Analystics is the base
8.5.6 NY Food Product Database
We use data the NY Food Product Database to identify businesses that sell New York branded products (lrfs==1).
Bean processors include any manufacturer that lists the following values for Sub Category: “Meat”.
We do not rely on the MWBE certification from this database as it is unclear whether is it for the processor or wholesaler. Rather we use the list of businesses certified MWBE by NYC and NYC.
8.5.6.1 Import and clean data
8.5.6.2 Join to SimplyAnalytics
Here we take the NY food product data base wholesale data information and join it with the Simply Analytics data.
8.5.7 MWBE
We join the NYS and NYC MWBE certified businesses with all of our data sets.
New York State provides a database with all New York State Contractors that are certified MWBE and by New York City. We download the entire directory from both.
We select the following commodity codes:
- 040 - ANIMALS, BIRDS, MARINE LIFE, AND POULTRY, INCLUDING ACCESSOR
- 160 - BUTCHER SHOP AND MEAT PROCESSING EQUIPMENT
- 375 - FOODS: BAKERY PRODUCTS (FRESH)
- 385 - FOODS, FROZEN
- 390 - FOODS: PERISHABLE
- 393 - FOODS: STAPLE GROCERY AND GROCER’S MISCELLANEOUS ITEMS
8.5.7.1 Import data
8.5.7.2 Join to FSIS
8.5.7.3 Join to Auctions
8.5.7.4 Join to SimplyAnalytics
8.5.7.5 Join to NY Food Product database
We match MWBE to NY food database processor data and find one match.
All NYC purchasing data from a wholesaler will be in the SimplyAnalytics data.
8.5.8 Interim Final data
Here we keep wholesaler data from SimplyAnalytics (with any additional information from other data sets included) and processor data from all other data sources.
Here we add in the columns for the address data that will be added manually. At this step, the data is saved as an excel file and then addresses and other missing information are manually added by the qualitative team. After these additions, the data is imported, geocoded and then exported in its final format.
8.5.9 Processed data
The data that was saved in the step above was sent to the qualitative team and they manually added addresses, indicated those businesses that do not sell beans, added in data on the businesses location along the supply chain where missing (notably if the business was a primary or secondary processor), and estimated values for max inventory capacity.
We add the wholesale data that are MWBE to the data.
8.5.9.1 Import and clean
8.5.9.2 Add in scale data
We use data from surveys conducted by Cornell of Slaughterhouses in the Northeast.
The scale is the maximum number of beef/dairy cattle that can be slaughtered/processed each day.
8.5.9.3 Geocode
8.5.9.4 Re-geocode missing with ArcGIS
For addresses that the Census geocoder could not resolve, we use ArcGIS as a fallback via tidygeocoder. Results are merged back into the main data frame.
8.5.9.5 Save
Here we save a full version and the version that can be shared publicly with business names and addresses removed. We also drop any columns that are not relevant.
8.5.9.5.1 Full data
8.6 Markets
8.6.1 Global
We assume global demand is unlimited.
8.6.2 Differentiated
Final data: Total demand for New York State differentiated dry beans.
To determine to total market for NYS differentiated beans, four dry bean growers with food grade cleaning capacity who sell directly to buyers were contacted (interviews, emails, phone calls). Sales data was provided by the producer. (Numbers were confirmed to make sure they are not double counted in the totals).
8.6.3 Institutional purchasing in New York State
We use data from NYC Food Policy, Good Food Purchasing data from 2019-2023, we download the full purchasing data set.
We provide average and standard deviation for price per lb. and kg. for dry beans by form (canned, dried, frozen), and by agency from 2019-2023. We drop data points where the price is more than two standard deviations outside of the average price.
8.6.4 Import data
Here we import data and define variables.
8.6.5 Understand missing total weight data
The missing data does have the number of units, but does not have weight in lbs. We can use the information provided about the product and estimate how much the product weighed and estimate a total weight.
Department of Education has a large purchase that lists number of unit. Units are #10 cans, we assume they each weight 6 lbs 15 oz (111 oz).
For now, we are not going to take this step, but can in the future if needed.
8.6.6 Summary
We removed outliers that were more than 2 standard deviations from the mean based on price per pound, within agency, year, and form. We dropped 17 observations (397 down to 380).
Data are pulled for purchases with no attributes that would allow for a higher price (i.e., not MWBE, not purchased from New York). Because we will run the model with different scenarios related to local and MWBE, the data of interest are the bean purchases of non-local and non-MWBE products (i.e., no premium, defined as NY State spend y/n==N and MWBE y/n == N). Note there are no purchases of beans from MWBE businesses in the data.
8.6.6.1 Figure - Business as usual
We group all agencies together and show average price per pound by type (bagged, canned). We change the name dried to bagged to be consistent with other figures.
8.7 LCA
Jasmine and Elsie add.