Skip to content

perf(calendar): pre-filter ICS text before parsing#4168

Draft
KristjanESPERANTO wants to merge 1 commit into
MagicMirrorOrg:developfrom
KristjanESPERANTO:test-calendar
Draft

perf(calendar): pre-filter ICS text before parsing#4168
KristjanESPERANTO wants to merge 1 commit into
MagicMirrorOrg:developfrom
KristjanESPERANTO:test-calendar

Conversation

@KristjanESPERANTO
Copy link
Copy Markdown
Collaborator

This adds a pre-filter before parsing calendar ICS data. It drops simple one-off events that are clearly outside the configured date range, so node-ical has less work to do on large calendars.

Recurring events, overrides, RDATE, and unclear cases are kept and still handled by the normal parser logic.

I’ve spent quite a bit of time tightening this up because calendar data can be tricky. The pre-filter is intentionally conservative: when it is not sure, it keeps the event and lets the existing parser handle it.

Why? Some calendars contain years of old non-recurring events. Parsing all of them is unnecessary when MagicMirror only needs the configured visible window.

Doubts: The whole thing is a bit complex, and I'm wondering whether it makes sense to merge it. There have been hardly any complaints, and sooner or later, we won't be supporting slow Raspberry Pis anymore anyway (see #3796). Is it worth increasing the complexity of our codebase?

Solves #4103.

Drop simple one-off events outside the configured date window before
passing the ICS text to node-ical. Recurring events and overrides are
kept, because their actual instances need the normal expansion logic.

The pre-filter is conservative: if an event cannot be classified cheaply,
it stays in the feed and the parser handles it as before. This keeps the
optimization local while reducing parser work for calendars with many old
non-recurring events.

Ref MagicMirrorOrg#4103
@rejas
Copy link
Copy Markdown
Collaborator

rejas commented May 25, 2026

Just a thought: would a library like https://github.com/runely/ics-filter offload some complexity from this PR?

@sdetweil
Copy link
Copy Markdown
Collaborator

sdetweil commented May 25, 2026

@rejas that filters out ALL past events (we want SOME, sometimes)

the filter will only filter out past events

@sdetweil
Copy link
Copy Markdown
Collaborator

i also replied in the issue, not in the PR.

@KristjanESPERANTO
Copy link
Copy Markdown
Collaborator Author

Thanks @rejas for the pointer to ics-filter! I had a closer look and did some tests - I think the general approach of that package fits well, though there are a few edges. I just opened a first PR there to address one of them (case-insensitive property names): runely/ics-filter#1

I'm setting this to draft for now and waiting for the author to respond. I'd rather improve an existing package than maintain our own implementation here :)

@KristjanESPERANTO KristjanESPERANTO marked this pull request as draft May 25, 2026 21:24
@KristjanESPERANTO
Copy link
Copy Markdown
Collaborator Author

that filters out ALL past events (we want SOME, sometimes)

We can pass now - maximumNumberOfDays as the min date to icsFilter(), which is the same window filterEvents() (in ./defaultmodules/calendar/calendarfetcherutils.js) already uses for past events.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants