Skip to content

fix(imap): detect iMIP when method= is missing or mime is application/ics#12790

Open
lubo92 wants to merge 3 commits into
nextcloud:mainfrom
lubo92:fix/imip-detection-without-method-param
Open

fix(imap): detect iMIP when method= is missing or mime is application/ics#12790
lubo92 wants to merge 3 commits into
nextcloud:mainfrom
lubo92:fix/imip-detection-without-method-param

Conversation

@lubo92
Copy link
Copy Markdown

@lubo92 lubo92 commented Apr 22, 2026

Summary

Two upstream-detection gaps let calendar invitations slip through unprocessed:

1. Proton Mail Bridge strips the method= parameter. During E2E re-assembly Bridge loses the Content-Type parameter on the text/calendar part:

Content-Type: text/calendar; filename="invite.ics"; name="invite.ics"

instead of

Content-Type: text/calendar; charset=UTF-8; method=REQUEST

The ICS body still contains METHOD:REQUEST, but `MessageMapper::getBodyStructureData()` and `ImapMessageFetcher::getPart()` both key off the Content-Type parameter and ignore the body. Result: every single Proton-Bridge user silently loses inbound invitations from Gmail, M365, Fastmail, …

2. application/ics is ignored. Google's invitations ship two calendar parts — one text/calendar, one application/ics. If the calling code hands the fetcher the application/ics part (some MUAs/filters prefer the alphabetically-second attachment), it is treated as a regular attachment and no scheduling entry is built.

Changes

  • `MessageMapper::getBodyStructureData()`: accepts both mime types as iMIP candidates, drops the method= precondition (the method resolution happens further down).
  • `ImapMessageFetcher::getPart()`: accepts both mime types; if the Content-Type has no method= parameter, parses `METHOD:` from the ICS body instead. Body is only loaded in the fallback path, so the common case keeps its I/O pattern unchanged.

Tests

Two new fixtures + data-provider entries in `MessageMapperTest::testGetBodyStructureIsImipMessage()`:

  • `request_proton_bridge.txt` — text/calendar without method= parameter, METHOD:REQUEST in body
  • `request_application_ics.txt` — application/ics instead of text/calendar

Both must now evaluate to `isImipMessage() === true`.

Real-world impact

Empirically verified on a production Proton-Business + Nextcloud 32 + Mail-App 5.7.12 setup: before this change, Gmail and M365 invitations delivered via Bridge never reached CalDAV. After applying the same fix in-place, the iMIP pipeline works end-to-end for requests, replies, and cancels. Repro MIME samples available on request.

Checklist

  • Conventional-Commits commit message
  • DCO signed-off commit
  • Unit tests for new branch and fixture files added
  • No behaviour change for messages that do carry a valid method= parameter

Co-authored-with Claude Opus 4.7.

@SebastianKrupinski
Copy link
Copy Markdown
Contributor

SebastianKrupinski commented Apr 23, 2026

Hi @lubo92

Thanks for the PR!

I think the method discovery can actually be removed, the back end code no longer has split handling for calendar invitations, like it did before,

https://github.com/nextcloud/server/blob/a3e6ea0e836cbb554c87af18924b50d05f25786a/lib/private/Calendar/Manager.php#L229

There for the external (pre ics parsing) method discovery in the mail app, no longer really has a use. The method discovery now happens after the calendar parses the event.

https://github.com/nextcloud/server/blob/a3e6ea0e836cbb554c87af18924b50d05f25786a/apps/dav/lib/CalDAV/CalendarImpl.php#L232

The only thing I would leave is the ics file discovery based on type text/calendar and application/ics

@kesselb what do you think?

@kesselb
Copy link
Copy Markdown
Contributor

kesselb commented Apr 23, 2026

Thanks for your PR!

We've discussed this a while ago: #11009

I reached out to Proton back then, and they confirmed the bug and told me they would add it to their roadmap. I would kindly ask you to reach out and flag it again, as that hopefully raises the chances of having this issue resolved properly at some point. My request ticket number was 3648275.


Why did you also add application/ics? The RFC is quite strict on this, it should be text/calendar. Is there a provider generating those with application/ics?


I think we still need the code path, as it adds the invitation to the scheduling array which is later used to render the iMIP component. I'm not aware that we want to get rid of it.

$allContentTypeParameters = $p->getAllContentTypeParameters();
if ($p->getType() === 'text/calendar') {
// Handle event data like a regular attachment
// Outlook doesn't set a content disposition
// We work around this by checking for the name only
if ($p->getName() !== null) {
$this->attachments[] = [
'id' => $p->getMimeId(),
'messageId' => $this->uid,
'fileName' => $p->getName(),
'mime' => $p->getType(),
'size' => $p->getBytes(),
'cid' => $p->getContentId(),
'disposition' => $p->getDisposition()
];
}
// return if this is an event attachment only
// the method parameter determines if this is a iMIP message
if (!isset($allContentTypeParameters['method'])) {
return;
}
if (in_array(strtoupper($allContentTypeParameters['method']), ['REQUEST', 'REPLY', 'CANCEL'])) {
$this->scheduling[] = [
'id' => $p->getMimeId(),
'messageId' => $this->uid,
'method' => strtoupper($allContentTypeParameters['method']),
'contents' => $this->loadBodyData($p, $partNo, $isFetched),
];
return;
}
}

@SebastianKrupinski
Copy link
Copy Markdown
Contributor

Why did you also add application/ics? The RFC is quite strict on this, it should be text/calendar. Is there a provider generating those with application/ics?

A lot of providers add this.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 7, 2026

Hello there,
Thank you so much for taking the time and effort to create a pull request to our Nextcloud project.

We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process.

Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6

Thank you for contributing to Nextcloud and we hope to hear from you soon!

(If you believe you should not receive this message, you can add yourself to the blocklist.)

…/ics

Two upstream-detection gaps let calendar invitations slip through:

1. Proton Mail Bridge strips the `method=` parameter from the
   text/calendar Content-Type header while re-assembling a message
   after E2E-decryption. `MessageMapper::getBodyStructureData()`
   requires that parameter to flag the message as iMIP, and
   `ImapMessageFetcher::getPart()` requires it to populate the
   scheduling list — so every Bridge-delivered invitation is
   silently dropped.

2. Google's calendar invitations occasionally ship the same event
   twice: once as text/calendar and once as application/ics.
   Upstream ignored the application/ics part entirely, which means
   a mail client fed the "wrong" part (some MUAs prefer the second
   attachment) saw nothing.

This commit widens both detection sites to also accept
application/ics, and in the fetcher falls back to the METHOD:
line inside the ICS body when the Content-Type parameter is
missing. The body is only loaded in the fallback path, so the
common case (method= present) keeps the same I/O pattern it had
before.

Added two parameterised test fixtures:
  - request_proton_bridge.txt (text/calendar, no method= param)
  - request_application_ics.txt (application/ics)

Both are expected to now be flagged as iMIP.

AI-assisted: Claude Code (Claude Opus 4.7)
Signed-off-by: Wolfgang Lubowski <w.lub92@gmail.com>
@lubo92 lubo92 force-pushed the fix/imip-detection-without-method-param branch from a4351f8 to 7c15528 Compare May 12, 2026 15:30
@lubo92
Copy link
Copy Markdown
Author

lubo92 commented May 12, 2026

Thanks @SebastianKrupinski and @kesselb for the careful read. Quick update plus answers below.

Update on the PR itself

  • Rebased on main.
  • Fixed the psalm RedundantCondition failure (is_string($contents) was redundant — loadBodyData() is : string-typed).
  • Added the AI-assisted: trailer required by AGENTS.md.

On scope (re @SebastianKrupinski)

Agreed that the calendar backend now does its own METHOD discovery after ICS parsing in Calendar/Manager.php:229 / CalDAV/CalendarImpl.php:232. But the $this->scheduling[] array populated here in ImapMessageFetcher::getPart() isn't consumed by the server calendar — it's exposed via ImapMessageFetcher::getScheduling() and fed into the mail app's own iMIP UI rendering (the pointer @kesselb gave at ImapMessageFetcher.php:306-338). Removing the path wholesale would regress the in-app iMIP component.

So I kept the codepath and only widened the discovery in two minimally invasive ways:

  1. Accept application/ics alongside text/calendar at both detection sites.
  2. When the Content-Type method= parameter is missing, parse the METHOD: line out of the ICS body once — body data is only loaded in the fallback path, so the common case has identical I/O to before.

Happy to revisit if you'd rather drop the REQUEST/REPLY/CANCEL filter as well and let the backend decide — but it felt safer to keep the filter as-is for this fix.

On application/ics (re @kesselb)

You're right that RFC 5546 is strict on text/calendar. The in-the-wild reality differs: Google's invitations occasionally ship the same event twice — once as text/calendar, once as application/ics. The existing tests/data/imip/request_google.txt fixture shows the dual-part pattern; the new request_application_ics.txt exercises the case where the application/ics part is the one the MUA prefers. We've seen the same application/ics-only delivery from a few smaller IMAP-fronted setups too.

On Proton (re @kesselb)

Will reach out to Proton again referencing ticket 3648275 — I agree the proper fix belongs upstream in Bridge. Treating this PR as the defensive fallback so Bridge users aren't silently losing invitations until that lands.

PTAL when you have a minute.

Comment thread lib/IMAP/ImapMessageFetcher.php Outdated
$contents = null;
if ($method === null) {
$contents = $this->loadBodyData($p, $partNo, $isFetched);
if (preg_match('/^METHOD:\s*(\S+)/mi', $contents, $m)) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (preg_match('/^METHOD:\s*(\S+)/mi', $contents, $m)) {
if (preg_match('/^METHOD:([A-Z]+)/mi', $contents, $m) === 1) {

@kesselb
Copy link
Copy Markdown
Contributor

kesselb commented May 12, 2026

Thanks for the update.

With the current information (provided by you and the original report), I am okay with looking into the attachment to check if it looks like an event invitation.

For a Google invitation damaged by Proton, the actual fix is obtaining the method from the attachment. Nothing else needed.

Looking at application/ics seems nice, but in this case it is a nice-to-have on the side. I would prefer to not do that. If someone reaches out with a provider that only includes application/ics, we can reevaluate (in case they are not fixing their implementation). That matches the current state in Thunderbird.

The existing tests/data/imip/request_google.txt fixture shows the dual-part pattern.

It does not. I created that sample based on an original Google invitation, but dropped all unnecessary parts including the base64 encoded application/ics part.

…ication/ics

Address review feedback from kesselb:
- drop application/ics handling in both detection sites
  (Thunderbird's stance — re-evaluate if a real-world provider that
  only ships application/ics shows up)
- tighten METHOD regex to /^METHOD:([A-Z]+)/mi with === 1 check
  (matches the iCalendar grammar: METHOD value is bare token, no
  whitespace allowed between colon and value per RFC 5545)
- remove the now-obsolete application/ics fixture + test row

NOTE for squash: original commit body still mentions application/ics
support. When squashing, please drop that section so the message only
covers the Proton Bridge / METHOD-from-body fix.

AI-assisted: Claude Code (Claude Opus 4.7)
Signed-off-by: Wolfgang Lubowski <w.lub92@gmail.com>
@lubo92
Copy link
Copy Markdown
Author

lubo92 commented May 13, 2026

@kesselb thanks. Pushed a fixup commit (e4990e6) that scopes the PR down per your guidance:

  • dropped the application/ics widening at both detection sites (ImapMessageFetcher + MessageMapper); we can revisit if a real-world report comes in for a provider that only ships application/ics
  • tightened the METHOD regex to '/^METHOD:([A-Z]+)/mi' with the === 1 check
  • removed the now-obsolete tests/data/imip/request_application_ics.txt fixture and its data-provider row; the Proton Bridge fixture stays as the regression case for the actual fix

Per AGENTS.md this went in as --fixup rather than amend. Heads-up for the squash: the original commit body and subject still mention application/ics — when squashing, please trim that down so the message only covers the Proton Bridge / METHOD-from-body fallback.

Will re-ping Proton with their ticket 3648275 in parallel so the proper Bridge-side fix gets attention again.

@lubo92
Copy link
Copy Markdown
Author

lubo92 commented May 13, 2026

FYI: the two failing checks on e4990e6 (Block fixup and squash commits + Block unconventional commits) both fire on the fixup! commit-subject prefix and are the "intentional and expected" failures per AGENTS.md — they'll clear automatically when the branch is rebased + autosquashed before merge. Substantive checks (DCO, lint, static analysis, tests) are still waiting on first-time-contributor approval.

…ication/ics

Add a regression test exercising ImapMessageFetcher::getPart()'s
METHOD-from-body fallback (which is the new branch introduced by this
PR). Uses the existing request_proton_bridge fixture and the same
saveMimeMessage / fetchMessage pattern as the surrounding S/MIME
tests, so it goes through the real Horde+IMAP stack instead of just
parsing the MIME tree like the MessageMapperTest case does.

Asserts that the scheduling entry is emitted, its method is REQUEST,
and the contents we stash contain the METHOD: line we parsed.

AI-assisted: Claude Code (Claude Opus 4.7)
Signed-off-by: Wolfgang Lubowski <w.lub92@gmail.com>
@lubo92
Copy link
Copy Markdown
Author

lubo92 commented May 13, 2026

Pushed a second fixup (404bb93) adding an integration test for the METHOD-from-body fallback. Uses the existing request_proton_bridge.txt fixture and the saveMimeMessage / fetchMessage pattern from the surrounding S/MIME tests in ImapMessageFetcherIntegrationTest, so it exercises the new branch through the real Horde+IMAP stack — which is what the MessageMapperTest path doesn't reach.

This is what was driving the codecov patch-coverage failure (9.09%): the new if (preg_match…) line in ImapMessageFetcher::getPart() had no test exercising it before. Should clear codecov once CI runs.

Note: I don't have the integration-test docker stack locally so I couldn't verify the assertions; CI will be the source of truth.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants