Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EuroCrops get_label cannot handle features with None attributes #2497

Open
burakekim opened this issue Jan 3, 2025 · 0 comments · May be fixed by #2499
Open

EuroCrops get_label cannot handle features with None attributes #2497

burakekim opened this issue Jan 3, 2025 · 0 comments · May be fixed by #2499
Labels
datasets Geospatial or benchmark datasets

Comments

@burakekim
Copy link
Contributor

burakekim commented Jan 3, 2025

Description

The get_label method of the EuroCrops dataset is attempting to render features with None values, which raises an error. The attribute EC_hcat_c is set to None for some countries in the dataset, and this is not handled in the source code.

hcat_code = feature['properties'][self.label_name] # Here, self.label_name is "EC_hcat_c"

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[7], [line 1](vscode-notebook-cell:?execution_count=7&line=1)
----> [1](vscode-notebook-cell:?execution_count=7&line=1) eurocrops_sk[small_bbox]

File ~\Desktop\torchgeo\torchgeo\datasets\geo.py:759, in VectorDataset.__getitem__(self, query)
    [753](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/burak/Desktop/torchgeo/docs/tutorials/~/Desktop/torchgeo/torchgeo/datasets/geo.py:753)         for feature in src.filter(bbox=(minx, miny, maxx, maxy)):
    [754](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/burak/Desktop/torchgeo/docs/tutorials/~/Desktop/torchgeo/torchgeo/datasets/geo.py:754)             # Warp geometries to requested CRS
    [755](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/burak/Desktop/torchgeo/docs/tutorials/~/Desktop/torchgeo/torchgeo/datasets/geo.py:755)             # print("feature['properties']", feature['properties'])
    [756](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/burak/Desktop/torchgeo/docs/tutorials/~/Desktop/torchgeo/torchgeo/datasets/geo.py:756)             shape = fiona.transform.transform_geom(
    [757](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/burak/Desktop/torchgeo/docs/tutorials/~/Desktop/torchgeo/torchgeo/datasets/geo.py:757)                 src.crs, self.crs.to_dict(), feature['geometry']
    [758](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/burak/Desktop/torchgeo/docs/tutorials/~/Desktop/torchgeo/torchgeo/datasets/geo.py:758)             )
--> [759](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/burak/Desktop/torchgeo/docs/tutorials/~/Desktop/torchgeo/torchgeo/datasets/geo.py:759)             label = self.get_label(feature)
    [760](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/burak/Desktop/torchgeo/docs/tutorials/~/Desktop/torchgeo/torchgeo/datasets/geo.py:760)             shapes.append((shape, label))
    [762](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/burak/Desktop/torchgeo/docs/tutorials/~/Desktop/torchgeo/torchgeo/datasets/geo.py:762) # Rasterize geometries

File ~\Desktop\torchgeo\torchgeo\datasets\eurocrops.py:215, in EuroCrops.get_label(self, feature)
    [213](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/burak/Desktop/torchgeo/docs/tutorials/~/Desktop/torchgeo/torchgeo/datasets/eurocrops.py:213)     return self.class_map[hcat_code]
    [214](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/burak/Desktop/torchgeo/docs/tutorials/~/Desktop/torchgeo/torchgeo/datasets/eurocrops.py:214) print("hcat_code", hcat_code)
--> [215](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/burak/Desktop/torchgeo/docs/tutorials/~/Desktop/torchgeo/torchgeo/datasets/eurocrops.py:215) hcat_code_list = list(hcat_code)
    [216](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/burak/Desktop/torchgeo/docs/tutorials/~/Desktop/torchgeo/torchgeo/datasets/eurocrops.py:216) if all(c == '0' for c in hcat_code_list):
    [217](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/burak/Desktop/torchgeo/docs/tutorials/~/Desktop/torchgeo/torchgeo/datasets/eurocrops.py:217)     break

TypeError: 'NoneType' object is not iterable

Steps to reproduce

from torchgeo.datasets import EuroCrops

# Limit to a country with relatively small label size
# It needs to have None values inside, like Slovakia (scroll down): https://github.com/maja601/EuroCrops/wiki/Slovakia
class EuroCropsSK(EuroCrops):
    base_url = 'https://zenodo.org/records/8229128/files/'

    hcat_fname = 'HCAT2.csv'
    hcat_md5 = 'b323e8de3d8d507bd0550968925b6906'
    hcat_code_column = 'HCAT2_code'

    label_name = 'EC_hcat_c'

    filename_glob = '*_EC*.shp'

    filename_regex = r"""
        ^(?P<country>[A-Z]{2})
        (_(?P<region>[A-Z]+))?
        _
        (?P<date>\d{4})
        _
        (?P<suffix>EC(?:21)?)
        \.shp$
    """
    date_format = '%Y'
    zenodo_files: tuple[tuple[str, str], ...] = (
        ('SK_2021.zip', 'c7762b4073869673edc08502e7b22f01'),
    )


# Download the labels
eurocrops_sk = EuroCropsSK('./eurocrops_sk', download=True)

# Split the bounds into two and take the small one for fast debugging
small_bbox, big_bbox = eurocrops_sk.bounds.split(proportion=0.1, horizontal=True)

# Just curious
print(small_bbox.area)

# Trigger __getitem__
eurocrops_sk[small_bbox]

Version

'0.7.0.dev0'

@burakekim burakekim changed the title EuroCrops getitem handling of features with None attributes EuroCrops get_label cannot handle features with None attributes Jan 3, 2025
@adamjstewart adamjstewart added the datasets Geospatial or benchmark datasets label Jan 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datasets Geospatial or benchmark datasets
Projects
None yet
2 participants