I compile a similar list myself, sharing it here

[cristianoarbex]

Hi,

I do compile a list myself of assets in the SP500. I use it both for research and for developing strategies for investment funds.

My file has a different structure, and it is more detailed. The columns are:

• CODE: A unique identifier for each asset, mandatory.
• RIC: RIC stands for Reuters Identifier Code. Reuters tracks dead companies, meaning it is possible to download historical data from them up to when they existed.
• TICKER: Optional user-friendly identifier for the asset, i.e. a ticker.
• NAME: An optional name for the asset, i.e. a company full name.
• DATES1: All the dates when the asset was added to the index. Dates are written in the format YYYYMMDD and if the company was added more than once, the field is separated by "-" (hyphen), such as YYYYMMDD-YYYYMMDD-... The dates are given in chronological order.
• DATES2: All the dates when the asset was removed from the index. Dates are written in the format YYYYMMDD and if the company was removed more than once, the field is separated by "-" (hyphen), such as YYYYMMDD-YYYYMMDD-... The dates are given in chronological order.
• MARGIN: A 0-1 field indicating whether the asset uses margin accounts (futures contracts). I also work with them, but not applicable here.
• LEVERAGE: Default leverage level for that asset (applicable for futures contracts). Always 1 for regular assets.
• LOTSIZE: Default lot size specific for that asset.
• TAGS: All tags associated with that asset. If multiple tags it must be separated by hyphen (“-”).

If you want to expand your repository I can collaborate, I also keep track of NASDAQ100 and a few other indices.

One specific piece of data that I am looking for is the historical sector breakdown of the SP500. What was the percentage of each sector over time (daily? yearly? monthly?). This is hard to find!

[fja05680]

Hi cristianoarbex,

I can see how those additional columns could prove useful. Thanks for your offer to collaborate. If I had more time, I might take you up on it. As it stands, I barely have enough time to keep up the index as it is. I started a new demanding job about a year ago and my time is much more limited. Good luck with your projects.

Farrell

[cristianoarbex]

Sure, no problem. The file I sent can also be used as a sanity check in order to verify whether our two trackings match!

[fdijohn]

Hi cristianoarbex, I know I'm jumping in about a year and a couple moths late, but do you have a more current list? The list above is missing some additions, and deletions from the S&P 500, so I was wondering if you had a current copy?
Thanks
Frank

[cristianoarbex]

Dear @fdijohn, I do in fact have a very up-to-date copy. This latest year I got access to Polygon for a period, then to MarketStack. I used this opportunity to run a sanity check in all files, compiling ticker changes throughout the years, as well as missing additions and deletions from SP500. I have been keeping the sanity check as a shareable Google Sheets file.

[fja05680]

@cristianoarbex I removed the link to the google Sheets file. Likely, it's a violation of their terms of services to share information gathered from them. In any case, I don't want to have it shared here. I obtained permission for the original dataset that I use.

[cristianoarbex]

No worries @fja05680!

[fdijohn]

Hi Farrell! When you say you have obtained permissions for the original data set that you use, what dataset are you referring to? Can you provide a link? Also, is your dataset kept up to date? Thanks! Frank

…

On Wed, Dec 11, 2024 at 8:42 AM Farrell Aultman ***@***.***> wrote: @cristianoarbex <https://github.com/cristianoarbex> I removed the link to the google Sheets file. Likely, it's a violation of their terms of services to share information gathered from them. In any case, I don't want to have it shared here. I obtained permission for the original dataset that I use. — Reply to this email directly, view it on GitHub <#14 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJ3YLSHZYTZDFY4RNZK345D2FA6MZAVCNFSM6AAAAABTJGAR66VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNJTGMZTGMY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[fdijohn]

Hi Cristianor! Did you see Farrell Aulytman's reply to us? Frank

…

On Wed, Dec 11, 2024 at 6:57 AM cristianoarbex ***@***.***> wrote: Dear @fdijohn <https://github.com/fdijohn>, I do in fact have a very up-to-date copy. This latest year I got access to Polygon for a period, then to MarketStack. I used this opportunity to run a sanity check in all files, compiling ticker changes throughout the years, as well as missing additions and deletions from SP500. I have been keeping the sanity check as a shareable Google Sheets file: https://docs.google.com/spreadsheets/d/1XJ13nZ0S1QFmorF4Xtgg6M56NHxWmWp4B-5w6ltJMGg/edit?usp=sharing — Reply to this email directly, view it on GitHub <#14 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJ3YLSH2HLGNURDL2FHP5CT2FASB5AVCNFSM6AAAAABTJGAR66VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNJTGIZDIOA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[fja05680]

Hi Frank,

This is covered in my Readme file in the "Notes" section:
https://github.com/fja05680/sp500/blob/master/README.md

I emailed "Andreas Clenow" to obtain permissions. That correspondence is here:
https://github.com/fja05680/pinkfish/blob/master/pinkfish/analysis.py

Sorry I had to remove it, but I could get in trouble for knowingly allowing a link to protected material. See below:

"You must not use Market Data for any purpose not directly related to the Services or your
account. The Market Data may not be copied, reproduced, republished, uploaded, posted,
publicly displayed, encoded, translated, transmitted, or distributed in any way (including
"mirroring") to any other computer, server, website, or other medium for publication or
distribution or for any business or commercial enterprise, without Polygon's express prior
written consent."
https://cdn.sanity.io/files/dhlwe0i3/production/178888987016aefc45a441786ba1f5daa311d279.pdf

Farrell

[fdijohn]

Hi Farrell! I think I understand what you are saying, but wanted to confirm it with you. Cristiano wrote the following: << This latest year I got access to Polygon for a period, then to MarketStack. I used this opportunity to run a sanity check in all files, compiling ticker changes throughout the years, as well as missing additions and deletions from SP500. I have been keeping the sanity check as a shareable Google Sheets file.

>

Are you saying that because he had access to Polygon, and then MarketStack, and used the data from their services to "run a sanity check" on the data in the files, and then shared that file on GitHub, it is a violation of Polygon's agreement and could cause you to get into trouble if you left it there. Is that the essence of the issue? Thanks Frank

…

On Wed, Dec 11, 2024 at 10:38 AM Farrell Aultman ***@***.***> wrote: Hi Frank, This is covered in my Readme file in the "Notes" section: https://github.com/fja05680/sp500/blob/master/README.md I emailed "Andreas Clenow" to obtain permissions. That correspondence is here: https://github.com/fja05680/pinkfish/blob/master/pinkfish/analysis.py Sorry I had to remove it, but I could get in trouble for knowingly allowing a link to protected material. See below: "You must not use Market Data for any purpose not directly related to the Services or your account. The Market Data may not be copied, reproduced, republished, uploaded, posted, publicly displayed, encoded, translated, transmitted, or distributed in any way (including "mirroring") to any other computer, server, website, or other medium for publication or distribution or for any business or commercial enterprise, without Polygon's express prior written consent." https://cdn.sanity.io/files/dhlwe0i3/production/178888987016aefc45a441786ba1f5daa311d279.pdf Farrell — Reply to this email directly, view it on GitHub <#14 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJ3YLSBUVVRVZ7OQ3KAJ5332FBL5VAVCNFSM6AAAAABTJGAR66VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNJTGQ3TEOA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[fja05680]

Hi Frank,

Yes, if he used data from those services to make corrections to his list and then shared that list, my guess is that it violates the terms of service of those providers. I'm not judging, it's just best for me that it's not posted here.

Farrell