-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Equipment: Integration into modules #1341
Conversation
…as equipment in MNH modules
diff --git src/tlo/methods/breast_cancer.py src/tlo/methods/breast_cancer.py index 402266516..57e842119 100644 --- src/tlo/methods/breast_cancer.py +++ src/tlo/methods/breast_cancer.py @@ -646,8 +646,9 @@ class HSI_BreastCancer_Investigation_Following_breast_lump_discernible(HSI_Event self.TREATMENT_ID = "BreastCancer_Investigation" self.EXPECTED_APPT_FOOTPRINT = self.make_appt_footprint({"Over5OPD": 1, "Mammography": 1}) self.ACCEPTED_FACILITY_LEVEL = '3' # Biopsy only available at level 3 and above. - # TODO: but the appt footprints suggests mammography to be provided - self.EQUIPMENT = {'Slice Master sample processing Unit', 'Paraffin Dispense', 'Whatever used with biopsy'} + + # TODO: Eva's dummy equipment example (not sure if it actually needs to be added and if it is in the RF) + # {'Slice Master sample processing Unit', 'Paraffin Dispense', 'Whatever used with biopsy'} # ap_oct23 - Eva, I cannot locate a biopsy needle in the equipment - perhaps it is in consumables # the other equipment need is for histology in the lab - there is a whole long list of items needed @@ -771,8 +772,9 @@ class HSI_BreastCancer_StartTreatment(HSI_Event, IndividualScopeEventMixin): df.at[person_id, "brc_date_treatment"] = self.sim.date df.at[person_id, "brc_stage_at_which_treatment_given"] = df.at[person_id, "brc_status"] + # TODO: Eva's dummy equipment example - it needs to be replaced by real items from the RF # Update equipment - self.EQUIPMENT.update({'Anything used for mastectomy'}) + # self.EQUIPMENT.update({'Anything used for mastectomy'}) # Schedule a post-treatment check for 12 months: hs.schedule_hsi_event( diff --git src/tlo/methods/contraception.py src/tlo/methods/contraception.py index 6f9e4254b..8c9821b0c 100644 --- src/tlo/methods/contraception.py +++ src/tlo/methods/contraception.py @@ -1284,8 +1284,10 @@ class HSI_Contraception_FamilyPlanningAppt(HSI_Event, IndividualScopeEventMixin) # Update equipment if _new_contraceptive == 'female_sterilization': + # TODO: Eva's dummy example - needs to be replaced by real item(s) self.EQUIPMENT.update({'Smt used to sterilize a woman'}) elif _new_contraceptive == 'IUD': + # TODO: Eva's dummy example - needs to be replaced by real item(s) self.EQUIPMENT.update({'Equipment used when performing IUD'}) else:
diff --git src/tlo/methods/care_of_women_during_pregnancy.py src/tlo/methods/care_of_women_during_pregnancy.py index 2944ee41f..96a4b6975 100644 --- src/tlo/methods/care_of_women_during_pregnancy.py +++ src/tlo/methods/care_of_women_during_pregnancy.py @@ -572,7 +572,7 @@ class CareOfWomenDuringPregnancy(Module): def antenatal_care_scheduler(self, individual_id, visit_to_be_scheduled, recommended_gestation_next_anc): """ - This function is responsible for scheduling a womans next ANC contact in the schedule if she chooses to seek + This function is responsible for scheduling a woman's next ANC contact in the schedule if she chooses to seek care again. It is called by each of the ANC HSIs. :param individual_id: individual_id :param visit_to_be_scheduled: Number if next visit in the schedule (2-8) @@ -1572,8 +1572,8 @@ class HSI_CareOfWomenDuringPregnancy_SecondAntenatalCareContact(HSI_Event, Indiv self.module.antenatal_care_scheduler(person_id, visit_to_be_scheduled=3, recommended_gestation_next_anc=gest_age_next_contact) - # Then we administer interventions that are due to be delivered at this womans gestational age, which may be - # in addition to intervention delivered in ANC2 + # Then we administer interventions that are due to be delivered at this woman's gestational age, which may + # be in addition to intervention delivered in ANC2 if mother.ps_gestational_age_in_weeks < 26: self.module.albendazole_administration(hsi_event=self) self.module.iptp_administration(hsi_event=self) @@ -1621,7 +1621,7 @@ class HSI_CareOfWomenDuringPregnancy_ThirdAntenatalCareContact(HSI_Event, Indivi antenatal care contact (ANC3). It is scheduled by the HSI_CareOfWomenDuringPregnancy_SecondAntenatalCareContact for women who choose to seek additional ANC after their previous visit. It is recommended that this visit occur at 26 weeks gestation. This event delivers the interventions to women which are part of ANC3. Additionally interventions - that should be delivered according to a womans gestational age and position in her ANC schedule are delivered. + that should be delivered according to a woman's gestational age and position in her ANC schedule are delivered. Finally scheduling the next ANC contact in the occurs during this HSI along with admission to antenatal inpatient ward in the case of complications"""
…' into equipment/integration_in_modules # Conflicts: # resources/healthsystem/infrastructure_and_equipment/ResourceFile_Equipment.csv # src/scripts/healthsystem/equipment/equipment_catalogue.py # src/tlo/methods/breast_cancer.py # src/tlo/methods/care_of_women_during_pregnancy.py # src/tlo/methods/contraception.py # src/tlo/methods/healthsystem.py # src/tlo/methods/newborn_outcomes.py
…e_name before event_name, sorted by indexes; (2) added summary catal. (3) focused - only this cat. without empty equip rows and split by item per row
…e_name before event_name, sorted by indexes; (2) added summary catal. (3) focused - only this cat. without empty equip rows and split by item per row
…ration_in_modules # Conflicts: # src/scripts/healthsystem/equipment/equipment_catalogue.py
… cancer_consumables.py to prevent repeating code blocks
…ated spelling of cystoscopy within bladder_cancer.py and the associated resource file.
There are some conflicts in tb module, could someone (@tbhallett, @tdm32, ...?) resolve them? Thank you! |
@EvaJanouskova @tbhallett - got a bit lost with all the activity here. Am I still needed to add the ANC equipment packages to the HSIs here? |
Yes please. You can do it here: #1378. Thanks! |
…tructure # Conflicts: # src/tlo/methods/tb.py
We've resolved and pushed a merge commit to this branch, |
* equip: update lookup pkg names to consider option of having one item within multiple pkgs * RF_EquipCatalogue: pkgs defined * modules: usages of equip pkgs included * RF_EquipCatalogue: Endoscope and ECG added * equip: warning messages clarified * equip & test_equip: fix lookup_item_codes_from_pkg_name() fnc and test its functionality diff --git src/tlo/methods/equipment.py src/tlo/methods/equipment.py index 1759a994f..099226748 100644 --- src/tlo/methods/equipment.py +++ src/tlo/methods/equipment.py @@ -253,8 +253,17 @@ class Equipment: It is expected that this is used by the disease module once and then the resulting equipment item_codes are saved on the module.""" df = self.catalogue + item_codes = set() - if pkg_name not in df['Pkg_Name'].unique().split(", "): + item_codes.update(df.loc[df['Pkg_Name'] == pkg_name, 'Item_Code'].values) + + all_pkg_names = set(df['Pkg_Name'].unique()[~pd.isnull(df['Pkg_Name'].unique())]) + all_multiple_pkg_names = [name for name in all_pkg_names if ", " in name] + for multiple_pkg_name in all_multiple_pkg_names: + if pkg_name in multiple_pkg_name.split(", "): + item_codes.update(df.loc[df['Pkg_Name'] == multiple_pkg_name, 'Item_Code'].values) + + if item_codes: + return item_codes + else: raise ValueError(f'That Pkg_Name is not in the catalogue: {pkg_name=}') - - return set(df.loc[df['Pkg_Name'] == pkg_name, 'Item_Code'].values) diff --git tests/test_equipment.py tests/test_equipment.py index a02ea282f..a39124454 100644 --- tests/test_equipment.py +++ tests/test_equipment.py @@ -22,14 +22,18 @@ def test_core_functionality_of_equipment_class(seed): # Create toy data catalogue = pd.DataFrame( + # PkgWith0+1 stands alone or as multiple pkgs for one item; PkgWith1 is only as multiple pkgs + # for one item; PkgWith3 only stands alone [ {"Item_Description": "ItemZero", "Item_Code": 0, "Pkg_Name": 'PkgWith0+1'}, - {"Item_Description": "ItemOne", "Item_Code": 1, "Pkg_Name": 'PkgWith0+1'}, + {"Item_Description": "ItemOne", "Item_Code": 1, "Pkg_Name": 'PkgWith0+1, PkgWith1'}, {"Item_Description": "ItemTwo", "Item_Code": 2, "Pkg_Name": float('nan')}, + {"Item_Description": "ItemThree", "Item_Code": 3, "Pkg_Name": float('PkgWith3')}, ] ) data_availability = pd.DataFrame( - # item 0 is not available anywhere; item 1 is available everywhere; item 2 is available only at facility_id=1 + # item 0 is not available anywhere; item 1 is available everywhere; item 2 is available only at facility_id=1; + # availability not defined for item 3 [ {"Item_Code": 0, "Facility_ID": 0, "Pr_Available": 0.0}, {"Item_Code": 0, "Facility_ID": 1, "Pr_Available": 0.0}, @@ -134,17 +138,22 @@ def test_core_functionality_of_equipment_class(seed): # Lookup the item_codes that belong in a particular package. # - When package is recognised - assert {0, 1} == eq_default.lookup_item_codes_from_pkg_name(pkg_name='PkgWith0+1') # these items are in the same - # package + # if items are in the same package (once standing alone, once within multiple pkgs defined for item) + assert {0, 1} == eq_default.lookup_item_codes_from_pkg_name(pkg_name='PkgWith0+1') + # if the pkg within multiple pkgs defined for item + assert {1} == eq_default.lookup_item_codes_from_pkg_name(pkg_name='PkgWith1') + # if the pkg only stands alone + assert {3} == eq_default.lookup_item_codes_from_pkg_name(pkg_name='PkgWith3') + # - Error thrown when package is not recognised with pytest.raises(ValueError): eq_default.lookup_item_codes_from_pkg_name(pkg_name='') - equipment_item_code_that_is_available = [0, 1, ] equipment_item_code_that_is_not_available = [2, 3,] + def run_simulation_and_return_log( seed, tmpdir, equipment_in_init, equipment_in_apply ) -> Dict: * equip & test_equip: update lookup for equipment from pkgs to less "expensive" version, lookup fnc renamed to from_pkg_names() diff --git src/tlo/methods/equipment.py src/tlo/methods/equipment.py index 099226748..3a4fb24ba 100644 --- src/tlo/methods/equipment.py +++ src/tlo/methods/equipment.py @@ -1,6 +1,6 @@ import warnings from collections import defaultdict -from typing import Counter, Iterable, Literal, Set, Union +from typing import Counter, Dict, Iterable, Literal, Set, Union import numpy as np import pandas as pd @@ -77,6 +77,7 @@ class Equipment: # - Data structures for quick look-ups for items and descriptors self._item_code_lookup = self.catalogue.set_index('Item_Description')['Item_Code'].to_dict() + self._pkg_lookup = self._create_pkg_lookup() self._all_item_descriptors = set(self._item_code_lookup.keys()) self._all_item_codes = set(self._item_code_lookup.values()) self._all_fac_ids = self.master_facilities_list['Facility_ID'].unique() @@ -248,22 +249,37 @@ class Equipment: data=row.to_dict(), ) - def lookup_item_codes_from_pkg_name(self, pkg_name: str) -> Set[int]: - """Convenience function to find the set of item_codes that are grouped under a package name in the catalogue. - It is expected that this is used by the disease module once and then the resulting equipment item_codes are - saved on the module.""" - df = self.catalogue - item_codes = set() - - item_codes.update(df.loc[df['Pkg_Name'] == pkg_name, 'Item_Code'].values) - - all_pkg_names = set(df['Pkg_Name'].unique()[~pd.isnull(df['Pkg_Name'].unique())]) - all_multiple_pkg_names = [name for name in all_pkg_names if ", " in name] - for multiple_pkg_name in all_multiple_pkg_names: - if pkg_name in multiple_pkg_name.split(", "): - item_codes.update(df.loc[df['Pkg_Name'] == multiple_pkg_name, 'Item_Code'].values) - - if item_codes: - return item_codes + def from_pkg_names(self, pkg_names: Union[str, Iterable[str]]) -> Set[int]: + """Convenience function to find the set of item_codes that are grouped under requested package name(s) in the + catalogue.""" + # Make into a set if it is not one already + if isinstance(pkg_names, (str, int)): + pkg_names = set([pkg_names]) else: - raise ValueError(f'That Pkg_Name is not in the catalogue: {pkg_name=}') + pkg_names = set(pkg_names) + + item_codes = set() + for pkg_name in pkg_names: + if pkg_name in self._pkg_lookup.keys(): + item_codes.update(self._pkg_lookup[pkg_name]) + else: + raise ValueError(f'That Pkg_Name is not in the catalogue: {pkg_name=}') + + return item_codes + + def _create_pkg_lookup(self) -> Dict[str, Set[int]]: + df = self.catalogue + pkg_lookup = dict() + + pkg_names_raw = set(df['Pkg_Name'].unique()[~pd.isnull(df['Pkg_Name'].unique())]) + all_multiple_pkg_names = set(name for name in pkg_names_raw if ", " in name) + all_pkg_names = pkg_names_raw - all_multiple_pkg_names + for pkg_name in all_pkg_names: + pkg_lookup[pkg_name] = set(df.loc[df['Pkg_Name'] == pkg_name, 'Item_Code'].values) + for multiple_pkg_name in all_multiple_pkg_names: + for pkg_name in multiple_pkg_name.split(", "): + if pkg_name not in all_pkg_names: + pkg_lookup[pkg_name] = set() + all_pkg_names.update({pkg_name}) + pkg_lookup[pkg_name].update(set(df.loc[df['Pkg_Name'] == multiple_pkg_name, 'Item_Code'].values)) + return pkg_lookup diff --git tests/test_equipment.py tests/test_equipment.py index a39124454..0a462f08f 100644 --- tests/test_equipment.py +++ tests/test_equipment.py @@ -139,15 +139,15 @@ def test_core_functionality_of_equipment_class(seed): # Lookup the item_codes that belong in a particular package. # - When package is recognised # if items are in the same package (once standing alone, once within multiple pkgs defined for item) - assert {0, 1} == eq_default.lookup_item_codes_from_pkg_name(pkg_name='PkgWith0+1') + assert {0, 1} == eq_default.from_pkg_names(pkg_name='PkgWith0+1') # if the pkg within multiple pkgs defined for item - assert {1} == eq_default.lookup_item_codes_from_pkg_name(pkg_name='PkgWith1') + assert {1} == eq_default.from_pkg_names(pkg_name='PkgWith1') # if the pkg only stands alone - assert {3} == eq_default.lookup_item_codes_from_pkg_name(pkg_name='PkgWith3') + assert {3} == eq_default.from_pkg_names(pkg_name='PkgWith3') # - Error thrown when package is not recognised with pytest.raises(ValueError): - eq_default.lookup_item_codes_from_pkg_name(pkg_name='') + eq_default.from_pkg_names(pkg_names='') equipment_item_code_that_is_available = [0, 1, ] * equip lookup fnc renamed to from_pkg_names() * test_equip: fixes from rebase * [no ci] test_equip: test correct item codes returned for multiple specified pkgs * add ANC packages * edit equipment_availability_estimation.py to manually add cost category * upate ResourceFile_Equipment_Availability_Estimates.csv running new script --------- Co-authored-by: joehcollins <[email protected]> Co-authored-by: Tim Hallett <[email protected]>
@EvaJanouskova and @joehcollins think this is done, pending merge of #1298 |
src/scripts/data_file_processing/healthsystem/equipment/equipment_availability_estimation.py
Show resolved
Hide resolved
* add latest version of costing resource file * add latest version of costing resource file - with updated units for cost of consumables * update Gentamycin (units) * update item quantities for HIV prep and infant prep * update 1g to 1mg for Albendazole (Item code 52) * update units for gloves from 1 glove to "1 pair of gloves" * update item quantities for TB drugs add 3HP as new consumable for IPT * add conditions for each IPT recommendation * add prices for ARVs - and create placeholder for Isoniazid/Rifapentine * add Isoniazid/Rifapentine to `ResourceFile_Consumables_Items_and_Packages.csv` * add Isoniazid/Rifapentine to all relevant RFs - 1. `ResourceFile_Consumables_Items_and_Packages.csv` - list of item codes and names - 2. `ResourceFile_consumabes_matched.csv` - crosswalk between consumables in the TLO model and the OpenLMIS dataset to extract availability (used proxy 'Isoniazid, 100mg' OR 'Isoniazid, 300mg') - 3. `ResourceFile_Consumables_availability_small.csv` - Final availability estimates - updated by running `consumables_availability_estimation.py` - 4 `ResourceFile_Costing.xlsx` - cost of tablet based on external web report * Update to the functionality for requesting consumables across the MNH modules to allow for clearer requesting of units of consumables for costing (correct number of units not yet implemented for all consumables) * fixes to failing tests * linting * linting * change cotrimoxazole units to mg * merge in updated unit costs 3hp added and linked to consumables dataset * added dosage for amitriptyline antidepressant treatment * added dosage for epilepsy treatment * fix error in postnatal_supervisor.py leading to failing test * units added for care_of_women_during_pregnancy.py * units added for care_of_women_during_pregnancy.py * units added for labour.py * fix failing test. initial doses for newborn outcomes/postnatal supervisor * fix failing test. initial doses for newborn outcomes/postnatal supervisor * delete temporary ~ file created * update units of safety box to "1 disposed syringe (100 syringes per box)" * fix failing test. initial doses for newborn outcomes/postnatal supervisor * fixed error in labour. COPD consumable unites * remove consumable packages from diarrhoea.py * diarrhoea.py consumable units * cmd.py consumable units * rti.py consumable units * alri.py consumable units plus fixed error in diarrhoea.py * co: get item codes of consumables from item names and define number of units per case in the module (using new chosen units) * Add cystoscope, endoscope and prostate specific antigen test - add three consumables to `ResourceFile_Consumables_Items_and_Packages.csv` using the `generate_consumables_item_codes_and_packages.py` script - This is a replication of commit ff4d072 from PR #1341 * co: alternative consumables chosen by probs * co: 21 tablets per packet for both types of pills (informed by Emi) * co: update TODOs * cancer_cons: consumables updated and required units per case defined (as nmbs of chosen units) * [no ci] cancer_cons: rm outdated TODOs * [no ci] co: rm resolved TODOs * co: rm TODOs as opened an issue to resolve them (#1384) * cancers: missing arguments added * add availability data cancer consumables - Biopsy needle and Specimen container (This is based on assumptions made in `"05 - Resources/Module-healthsystem/consumables raw files/ResourceFile_hhfa_consumables.xlsx` * Revert "cancers: missing arguments added" This reverts commit 5e3d5c0. * cancer_cons: rm get_consumable_item_codes_cancers() argument * cancer_cons: names corrections -- item & consumables set * add depn to health system to copd * remove packing with dict * refactoring and adding todo * fix misnamed tb consumable * roll back file added by accident * test_co & co: alternatives rolled back * roll back second output * simplify _get_cons_group as no longer changing dose by age * remove todo, now resolved * linting and removed completed todo * linting --------- Co-authored-by: sm2511 <[email protected]> Co-authored-by: tdm32 <[email protected]> Co-authored-by: joehcollins <[email protected]> Co-authored-by: Tim Hallett <[email protected]> Co-authored-by: Eva Janouskova <[email protected]>
…tructure # Conflicts: # src/tlo/methods/cancer_consumables.py # src/tlo/methods/care_of_women_during_pregnancy.py # src/tlo/methods/copd.py # src/tlo/methods/labour.py # src/tlo/methods/newborn_outcomes.py
src/tlo/methods/prostate_cancer.py
Outdated
@@ -803,6 +804,7 @@ def __init__(self, module, person_id): | |||
self.TREATMENT_ID = "ProstateCancer_Investigation" | |||
self.EXPECTED_APPT_FOOTPRINT = self.make_appt_footprint({"Over5OPD": 1}) | |||
self.ACCEPTED_FACILITY_LEVEL = '1b' | |||
# todo @Eva - biopsy equipment needed (perhaps ultrasound to guide). histology lab equipment. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When I look at the commit f566709, seems this comment was removed there.
src/tlo/methods/prostate_cancer.py
Outdated
@@ -780,7 +781,7 @@ def apply(self, person_id, squeeze_factor): | |||
hsi_event=self | |||
) | |||
|
|||
# TODO: replace with PSA test when added to cons list | |||
# TODO: DISCUSSED - but to remain as todo -replace with PSA test when added to cons list |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
update comment?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this can be removed now.
src/tlo/methods/prostate_cancer.py
Outdated
@@ -733,6 +733,7 @@ def apply(self, person_id, squeeze_factor): | |||
) | |||
|
|||
# Check consumable availability | |||
# TODO: DISCUSSED - but to remain as todo -replace with PSA test when added to cons list |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove comment?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this can be removed now.
@@ -1294,6 +1294,8 @@ def apply(self, person_id, squeeze_factor): | |||
|
|||
self.get_consumables(item_codes=of_repair_cons) | |||
|
|||
# TODO DISCUSSED @Joe - add surgical package |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure which surgical package @joehcollins wanted to add - minor/major surgery?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added now!
src/tlo/methods/oesophagealcancer.py
Outdated
@@ -686,13 +686,15 @@ def apply(self, person_id, squeeze_factor): | |||
return hs.get_blank_appt_footprint() | |||
|
|||
# Check the consumables are available | |||
# todo: replace with endoscope? | |||
# todo: DISCUSS - add endoscope consumable to screening biopsy |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks like this done, so remove the comment?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@andrew-phillips-1 , @joehcollins, This is a bit confusing to me. The core items are currently screening_biopsy_core
, which is biopsy needle, but the todo says add endoscope, does it mean we want both, biopsy needle and endoscope for Investigation_Following_Dysphagia?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(@tbhallett do not remove this yet, it is not done.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will replace biopsy needle with endoscope, and wait for @joehcollins to confirm if it is what he wanted.
…py; bc: biopsy replaced with cystoscopy; oc biopsy replaced with endoscopy
Hi @joehcollins, we have a few thing which we hope you will help us with:
Thanks! |
Thanks again! |
Great, thank you @joehcollins. @tbhallett, this should be done. |
… out trailing/leading spaces)
This extracts the key changes to the declaration of equipment made so far, and pending items, from #1124
An initial run of the simulation at scale is
job_id=long_run_all_diseases-2024-05-13T141858Z