GET requests with certain keys/values combinations crash with HTTP 500

[CodeWithIsmail]

Bug Description

Several GET requests (and one POST) against /elements/area and related aggregation endpoints return HTTP 500 Internal Server Error instead of a 400 Bad Request, when the keys/values query parameters contain unusual but non-malicious content. I collected 19 such requests during an automated test session and reproduced every one of them manually in Postman with the same result.

General Information

• Tested against the live remote instance at https://api.ohsome.org/v1
• Version of the ohsome API: 1.10.4 . The main branch I read the source from is this commit
• Affected endpoint(s): /elements/area, /elements/area/groupBy/key, /elements/area/groupBy/boundary, /elements/area/ratio/groupBy/boundary, /elements/count/ratio/groupBy/boundary
• found via AutoRestTest, then manually reproduced in Postman

Error Reproduce command

1. curl -X GET https://api.ohsome.org/v1/elements/area?bcircles=23%3A8.625%2C49.3711%2C70000&keys=id%7Ctype%7Cversion%7Ctimestamp%7Cchangeset%7Cuser%7Cuid%7Cvisible%7Ctimestamp%7Cuser%7Cversion%7Cversion%7Cid&time=2020-01-01%2F2023-01-01%2FP1Y&types=way
2. curl -X GET https://api.ohsome.org/v1/elements/area/groupBy/key?bpolys=8.625%2C+49.3711%2C+8.7334%2C+49.4397%2C+8.7334%2C+49.3711%2C+8.625%2C+49.3711&groupByKeys=building%2Crailway%2Chighway&keys=highway%2Cbuilding%2Crailway&values=building%2Crailway%3Dstation%2Chighway%3Dprimary
3. curl -X GET https://api.ohsome.org/v1/elements/area?bboxes=15%3A8.625%2C49.3711%2C8.7334%2C49.4397%7C16%3A8.6%2C49.38%2C8.74%2C49.42%7C&format=csv&keys=id%7Ctype%7Cversion%7Ctimestamp%7Cchangeset%7Cuser%7Cuid%7Cvisible%7Ctimestamp%7Cuser%7Cversion%7Cversion%7Cid&showMetadata=true&time=2014-01-01%2F2017-01-01%2FP1Y
4. curl -X GET https://api.ohsome.org/v1/elements/area?bboxes=7%3A8.625%2C49.3711%2C8.7334%2C49.4397%7C8%3A8.6%2C49.38%2C8.74%2C49.42%7C&keys=version%7Ctimestamp%7Cchangeset%7Cuser%7Cuid%7Cvisible%7Ctimestamp%7Cuser%7Cversion&time=2014-01-01%2F2017-01-01%2FP1D&values=way
5. curl -X GET https://api.ohsome.org/v1/elements/area?bboxes=15%3A8.625%2C49.3711%2C8.7334%2C49.4397%7C16%3A8.6%2C49.38%2C8.74%2C49.42%7C&keys=id%7Ctype%7Cversion%7Ctimestamp%7Cchangeset%7Cuser%7Cuid%7Cvisible%7Ctimestamp%7Cuser&values=node
6. curl -X GET https://api.ohsome.org/v1/elements/area?bboxes=5%3A8.625%2C49.3711%2C8.7334%2C49.4397%7C6%3A8.6%2C49.38%2C8.74%2C49.42%7C&format=csv&keys=id%7Ctype%7Cversion%7Ctimestamp%7Cchangeset%7Cuser%7Cuid%7Cvisible%7Ctimestamp%7Cuser
7. curl -X POST https://api.ohsome.org/v1/elements/count/ratio/groupBy/boundary?bboxes=1%3A8.625%2C49.3711%2C8.7334%2C49.4397%7C2%3A8.625%2C49.3711%2C8.7334%2C49.4397&format=json&keys2=id&showMetadata=true&types=way&types2=node
8. curl -X GET https://api.ohsome.org/v1/elements/area?bboxes=9%3A8.625%2C49.3711%2C8.7334%2C49.4397%7C10%3A8.6%2C49.38%2C8.74%2C49.42%7C&keys=id%7Ctype%7Cversion%7Ctimestamp%7Cchangeset%7Cuser%7Cuid%7Cvisible%7Ctimestamp%7Cuser&showMetadata=false&values=relation
9. curl -X GET https://api.ohsome.org/v1/elements/area?bcircles=27%3A8.625%2C49.3711%2C30000&format=json&keys=id%7Ctype%7Cversion%7Ctimestamp%7Cchangeset%7Cuser%7Cuid%7Cvisible%7Ctimestamp%7Cuser%7Cversion%7Cversion%7Cid&showMetadata=false
10. curl -X GET https://api.ohsome.org/v1/elements/area?bboxes=3%3A8.625%2C49.3711%2C8.7334%2C49.4397%7C4%3A8.6%2C49.38%2C8.74%2C49.42%7C&format=csv&keys=version%7Ctimestamp%7Cchangeset%7Cuser%7Cuid%7Cvisible&timeout=20&types=way
11. curl -X GET https://api.ohsome.org/v1/elements/area?bboxes=13%3A8.625%2C49.3711%2C8.7334%2C49.4397%7C14%3A8.6%2C49.38%2C8.74%2C49.42%7C&keys=id%7Ctype%7Cversion%7Ctimestamp%7Cchangeset%7Cuser%7Cuid%7Cvisible%7Ctimestamp%7Cuser&showMetadata=false&types=way
12. curl -X GET https://api.ohsome.org/v1/elements/area?bcircles=25%3A8.625%2C49.3711%2C50000&format=geojson&keys=version%7Ctimestamp%7Cchangeset%7Cuser%7Cuid%7Cvisible%7Ctimestamp%7Cuser%7Cversion
13. curl -X GET https://api.ohsome.org/v1/elements/area?bboxes=3%3A8.625%2C49.3711%2C8.7334%2C49.4397%7C4%3A8.6%2C49.38%2C8.74%2C49.42%7C&format=geojson&keys=version%7Ctimestamp%7Cchangeset%7Cuser%7Cuid%7Cvisible&showMetadata=false
14. curl -X GET https://api.ohsome.org/v1/elements/area?bboxes=17%3A8.625%2C49.3711%2C8.7334%2C49.4397%7C18%3A8.6%2C49.38%2C8.74%2C49.42%7C&keys=id%7Ctype%7Cversion%7Ctimestamp%7Cchangeset%7Cuser%7Cuid%7Cvisible%7Ctimestamp%7Cuser%7Cversion%7Cversion%7Cid&timeout=40&types=way
15. curl -X GET https://api.ohsome.org/v1/elements/area/groupBy/boundary?bcircles=id1%3A8.625%2C49.3711%2C1000&format=csv&keys=&showMetadata=true&time=2022-01-01T12%3A00%3A00Z%2F2023-01-01T12%3A00%3A00Z%2FP1D
16. curl -X GET https://api.ohsome.org/v1/elements/area?bcircles=25%3A8.625%2C49.3711%2C50000&keys=id%7Ctype%7Cversion%7Ctimestamp%7Cchangeset%7Cuser%7Cuid%7Cvisible%7Ctimestamp%7Cuser%7Cversion%7Cversion%7Cid&time=2014-01-01%2F2017-01-01%2FP1Y&types=node
17. curl -X GET https://api.ohsome.org/v1/elements/area?bcircles=23%3A8.625%2C49.3711%2C70000&format=geojson&keys=version%7Ctimestamp%7Cchangeset%7Cuser%7Cuid%7Cvisible%7Ctimestamp%7Cuser%7Cversion&types=relation
18. curl -X GET https://api.ohsome.org/v1/elements/area/ratio/groupBy/boundary?keys=visible&keys2=highway&types2=node&bboxes=1%3A8.625%2C49.3711%2C8.7334%2C49.4397%7C2%3A8.625%2C49.3711%2C8.7334%2C49.4397
19. curl -X GET https://api.ohsome.org/v1/elements/area?bcircles=23%3A8.625%2C49.3711%2C70000&keys=id%7Ctype%7Cversion%7Ctimestamp%7Cchangeset%7Cuser%7Cuid%7Cvisible%7Ctimestamp%7Cuser%7Cversion%7Cversion%7Cid&time=2014-01-01%2F2017-01-01%2FP1Y&timeout=50&types=relation&values=node

Expected Behaviour

An unusual or malformed keys/values combination should come back as 400 Bad Request with a message describing what's wrong, the same way a malformed bboxes value already does; not a bare 500 with no explanation.

[Hagellach37]

Hey @CodeWithIsmail ,
we have seen a huge increase in number of requests for the ohsome API extraction endpoints over the past months. Hence, we had to restrict access to these endpoints.

At the same time we are moving towards ohsome API v2.0. This will come with user management and api keys to better control excessive requests.

I would suggest, that you create an API key and check out the new API version and adjust your scripts accordingly.

more details here: https://api.heigit.org/ohsome-api/v2-rc/docs#/