Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Copy data from [source] into the opportunity table(s) in the analytics db #2317

Closed
1 task done
mxk0 opened this issue Sep 30, 2024 · 0 comments · Fixed by #3228
Closed
1 task done

Copy data from [source] into the opportunity table(s) in the analytics db #2317

mxk0 opened this issue Sep 30, 2024 · 0 comments · Fixed by #3228
Assignees
@babebe

Comments

@mxk0
Copy link
Collaborator

mxk0 commented Sep 30, 2024 •
edited by babebe
Loading

Summary

Can be a full replace of the data already in the analytics db

Acceptance criteria

  • Analytics db opportunities have data
@mxk0 mxk0 added refinement infra topic: infra Infrastructure related tickets and removed infra labels Oct 6, 2024
@mxk0 mxk0 moved this from Icebox to Todo in Simpler.Grants.gov Product Backlog Oct 6, 2024
@mxk0 mxk0 added the topic: backend Backend development tickets label Oct 9, 2024
This was referenced Oct 9, 2024
Open
Closed
@mxk0 mxk0 removed the refinement label Oct 9, 2024
@mxk0 mxk0 removed topic: backend Backend development tickets topic: infra Infrastructure related tickets labels Nov 11, 2024
@mxk0 mxk0 changed the title Write a count of opportunities to the analytics database Copy date from [source] into the opportunity table(s) in the analytics db Nov 19, 2024
@mxk0 mxk0 changed the title Copy date from [source] into the opportunity table(s) in the analytics db Copy data from [source] into the opportunity table(s) in the analytics db Nov 19, 2024
@babebe babebe self-assigned this Dec 5, 2024
@babebe babebe moved this from Todo to In Progress in Simpler.Grants.gov Product Backlog Dec 9, 2024
@babebe babebe mentioned this issue Dec 16, 2024
Merged
@babebe babebe moved this from In Progress to In Review in Simpler.Grants.gov Product Backlog Dec 16, 2024
@babebe babebe linked a pull request Dec 17, 2024 that will close this issue
2317/Copy data from grants-db into the opportunity table(s) in the analytics db #3228
Merged
babebe added a commit that referenced this issue Dec 18, 2024
@babebe
2317/Copy data from grants-db into the opportunity table(s) in the an…
27a7a65 
…alytics db (#3228)

## Summary
Fixes #{[2317](#2317)}

### Time to review: __20 mins__

## Changes proposed
New Cli Function to upload opportunity tables into analytics db
S3 configuration to read `csv `opportunity tables 
Add S3 Environment variables 
Mock s3 client for testing
Added fixtures for AWS (From API code), test-schema and
opportunity-tables to aid in testing, scoped to test session.
Added fixture to delete table records after each test
Added opportunity table `csv` files for testing
Added test that checks files were successfully uploaded and records
inserted into test-schema tables



## Context for reviewers
> poetry run pytest
./tests/integrations/extracts/test_load_opportunity_data.py when running
test locally

## Additional information
> Screenshots, GIF demos, code examples or output to help show the
changes working as expected.
doug-s-nava pushed a commit that referenced this issue Dec 30, 2024
@babebe @doug-s-nava
2317/Copy data from grants-db into the opportunity table(s) in the an…
b429083 
…alytics db (#3228)

## Summary
Fixes #{[2317](#2317)}

### Time to review: __20 mins__

## Changes proposed
New Cli Function to upload opportunity tables into analytics db
S3 configuration to read `csv `opportunity tables 
Add S3 Environment variables 
Mock s3 client for testing
Added fixtures for AWS (From API code), test-schema and
opportunity-tables to aid in testing, scoped to test session.
Added fixture to delete table records after each test
Added opportunity table `csv` files for testing
Added test that checks files were successfully uploaded and records
inserted into test-schema tables



## Context for reviewers
> poetry run pytest
./tests/integrations/extracts/test_load_opportunity_data.py when running
test locally

## Additional information
> Screenshots, GIF demos, code examples or output to help show the
changes working as expected.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@babebe babebe

Labels
None yet
Projects
Simpler.Grants.gov Product Backlog
Status: Done
SimplerGrants Product Backlog (experiment to be deleted)
Status: Todo
Milestone
No milestone
2 participants
@mxk0 @babebe