Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

632 and 632+ bootstrap estimates #970

Open
topepo opened this issue Nov 24, 2024 · 1 comment
Open

632 and 632+ bootstrap estimates #970

topepo opened this issue Nov 24, 2024 · 1 comment
Labels
feature a feature request or enhancement work in progress something that has a prototype implementation

Comments

@topepo
Copy link
Member

topepo commented Nov 24, 2024

These take bootstrap resampling and use more sophisticated estimates to estimate and correct for bias. I have a prototype implementation used in aml4td in the 632-estimate branch.

Notes:

  • It does an internal permutation calculation to get the no-information rate. Right now I do 50 shuffles.
  • It would require that apparent = TRUE is used when making the rset.
  • There are some API decisions to make for the points above. Do we parameterize the number of shuffles, etc.
@topepo topepo added feature a feature request or enhancement work in progress something that has a prototype implementation labels Nov 24, 2024
@topepo
Copy link
Member Author

topepo commented Nov 24, 2024

Also related is #917, which I also have in the book and an implementation for.

@topepo topepo mentioned this issue Nov 24, 2024
Closed
@topepo topepo moved this to Backlog in Dangling Projects Nov 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature a feature request or enhancement work in progress something that has a prototype implementation
Projects
Dangling Projects
Status: Backlog
Milestone
No milestone
Development

No branches or pull requests
1 participant
@topepo