Missing support for libgit2's custom_headers makes using with Bitbucket Personal Access Tokens impossible without knowing username

[beamerblvd]

I have a pull request to submit shortly after I post this issue.

We have a Bitbucket Data Center installation and our users exclusively use Bitbucket Personal Access Tokens (there are no passwords on the site, users authenticate with hardware tokens). While it is technically possible to make this work, generically, with pygit2 by way of specifying the username and PAT as the username and password in a UserPass credential, in our specific dev environment, we don't always know the username, and our infrastructure team has specifically blocked Basic authentication and requires Bearer authentication, so it wouldn't matter if we knew the username anyway.

While we have admittedly made things difficult on ourselves and put ourselves in a situation where pygit2 is impossible to use against our Bitbucket installation, libgit2 technically supports this with its custom_headers feature, and there's no reason pygit2 can't be improved to support it, too.

To start off with, here are some work-arounds that I tried based on flawed suggestions from various AI agents:

Attempt 1:

In [1]: import pathlib, pygit2

In [2]: callbacks = pygit2.RemoteCallbacks(pygit2.UserPass('x-token-auth', 'BBDC-abc123def456ghi789+'))

In [3]: clone_dir = pathlib.Path("./bs_clone").resolve()

In [4]: clone_dir
Out[4]: PosixPath('/foo/bar/baz/bs_clone')

In [5]: pygit2.clone_repository(url="https://www.example.com/bitbucket/scm/project/repo.git", path=clone_dir, callbacks=callbacks)
---------------------------------------------------------------------------
GitError                                  Traceback (most recent call last)
Cell In[5], line 1
----> 1 pygit2.clone_repository(url="https://www.example.com/bitbucket/scm/project/repo.git", path=clone_dir, callbacks=callbacks)

File /foo/bar/baz/.venv/lib/python3.10/site-packages/pygit2/__init__.py:541, in clone_repository(url, path, bare, repository, remote, checkout_branch, callbacks, depth, proxy)
   539             crepo = ffi.new('git_repository **')
   540             err = C.git_clone(crepo, to_bytes(url), to_bytes(path), opts)
--> 541             payload.check_error(err)
   543 # Ok
   544 return Repository._from_c(crepo[0], owned=True)

File /foo/bar/baz/.venv/lib/python3.10/site-packages/pygit2/callbacks.py:111, in Payload.check_error(self, error_code)
   106 elif self._stored_exception is not None:
   107     # A callback mapped to a C function returning void
   108     # might still have raised an exception.
   109     raise self._stored_exception
--> 111 check_error(error_code)

File /foo/bar/baz/.venv/lib/python3.10/site-packages/pygit2/errors.py:67, in check_error(err, io)
    64     raise StopIteration()
    66 # Generic Git error
---> 67 raise GitError(message)

GitError: too many redirects or authentication replays

Attempt 2:

In [6]: pygit2.Config.get_global_config().set_multivar("http.extraHeader", "^$", "Authorization: Bearer BBDC-abc123def456ghi789+")

In [7]: repo = pygit2.clone_repository(url="https://www.example.com/bitbucket/scm/project/repo.git", path=clone_dir)
---------------------------------------------------------------------------
GitError                                  Traceback (most recent call last)
Cell In[7], line 1
----> 1 repo = pygit2.clone_repository(url="https://www.example.com/bitbucket/scm/project/repo.git", path=clone_dir)

File /foo/bar/baz/.venv/lib/python3.10/site-packages/pygit2/__init__.py:541, in clone_repository(url, path, bare, repository, remote, checkout_branch, callbacks, depth, proxy)
   539             crepo = ffi.new('git_repository **')
   540             err = C.git_clone(crepo, to_bytes(url), to_bytes(path), opts)
--> 541             payload.check_error(err)
   543 # Ok
   544 return Repository._from_c(crepo[0], owned=True)

File /foo/bar/baz/.venv/lib/python3.10/site-packages/pygit2/callbacks.py:111, in Payload.check_error(self, error_code)
   106 elif self._stored_exception is not None:
   107     # A callback mapped to a C function returning void
   108     # might still have raised an exception.
   109     raise self._stored_exception
--> 111 check_error(error_code)

File /foo/bar/baz/.venv/lib/python3.10/site-packages/pygit2/errors.py:67, in check_error(err, io)
    64     raise StopIteration()
    66 # Generic Git error
---> 67 raise GitError(message)

GitError: remote authentication required but no callback set

Attempt 3:

In [15]: def create_remote(repo, name, url):
   ...:     remote = repo.remotes.create(name, url)
   ...:     repo.config.set_multivar("http.extraHeader", "^$", "Authorization: Bearer BBDC-abc123def456ghi789+")
   ...:     return remote
   ...:

In [16]: repo = pygit2.clone_repository(url="https://www.example.com/bitbucket/scm/project/repo.git", path=clone_dir, remote=create_remote)
---------------------------------------------------------------------------
GitError                                  Traceback (most recent call last)
Cell In[16], line 1
----> 1 repo = pygit2.clone_repository(url="https://www.example.com/bitbucket/scm/project/repo.git", path=clone_dir, remote=create_remote)

File /foo/bar/baz/.venv/lib/python3.10/site-packages/pygit2/__init__.py:541, in clone_repository(url, path, bare, repository, remote, checkout_branch, callbacks, depth, proxy)
   539             crepo = ffi.new('git_repository **')
   540             err = C.git_clone(crepo, to_bytes(url), to_bytes(path), opts)
--> 541             payload.check_error(err)
   543 # Ok
   544 return Repository._from_c(crepo[0], owned=True)

File /foo/bar/baz/.venv/lib/python3.10/site-packages/pygit2/callbacks.py:111, in Payload.check_error(self, error_code)
   106 elif self._stored_exception is not None:
   107     # A callback mapped to a C function returning void
   108     # might still have raised an exception.
   109     raise self._stored_exception
--> 111 check_error(error_code)

File /foo/bar/baz/.venv/lib/python3.10/site-packages/pygit2/errors.py:67, in check_error(err, io)
    64     raise StopIteration()
    66 # Generic Git error
---> 67 raise GitError(message)

GitError: remote authentication required but no callback set

But with my changes in the soon-to-be-linked pull request, it works great:

In [1]: import pathlib, pygit2

In [2]: from typing import override

In [3]: class BitbucketCallbacks(pygit2.RemoteCallbacks):
   ...:     @override
   ...:     def custom_headers(self) -> list[str] | None:
   ...:         return ["Authorization: Bearer BBDC-abc123def456ghi789+"]
   ...: 

In [4]: clone_dir = pathlib.Path("./bs_clone").resolve()

In [5]: clone_dir
Out[5]: PosixPath('/foo/bar/baz/bs_clone')

In [6]: repo = pygit2.clone_repository(url="https://www.example.com/bitbucket/scm/project/repo.git", path=clone_dir, callbacks=BitbucketCallbacks())

In [7]: repo
Out[7]: pygit2.Repository('/foo/bar/baz/bs_clone/.git/')

In [9]: list(repo.branches)
Out[9]: 
['main-3.4.0',
 'origin/1',
 'origin/1.0.0',
 ...]

In [10]: branch = repo.branches.local.create("nicktest-3.4.0", repo[repo.branches["main-3.4.0"].target])

In [11]: branch.is_checked_out()
Out[11]: False

In [14]: repo.set_head(branch.name)

In [15]: branch.is_checked_out()
Out[15]: True

In [16]: (clone_dir / "nicktest.txt").write_text("testing 1 2 3\n")
Out[16]: 13

In [17]: repo.status()
Out[17]: {'nicktest.txt': <FileStatus.WT_NEW: 128>}

In [19]: repo.index.add("nicktest.txt")

In [20]: repo.index.write()

In [21]: repo.status()
Out[21]: {'nicktest.txt': <FileStatus.INDEX_NEW: 1>}

In [22]: tree_id = repo.index.write_tree()

In [23]: author = pygit2.Signature('Nick Williams', 'nick.williams@example.com')

In [24]: parents = [repo.head.target]

In [25]: commit_id = repo.create_commit("HEAD", author, author, "#1234: This is just a test", tree_id, parents)

In [30]: repo.remotes["origin"].push([f"{repo.head.name}:{repo.head.name}"], callbacks=BitbucketCallbacks())

In [31]: branch.upstream = repo.branches.remote["origin/nicktest-3.4.0"]

In [32]: branch.upstream_name
Out[32]: 'refs/remotes/origin/nicktest-3.4.0'

In [37]: repo.status()
Out[37]: {}

In [38]: repo.remotes["origin"].fetch(callbacks=BitbucketCallbacks())
Out[38]: <pygit2.remotes.TransferProgress at 0x11127d5d0>

In [39]: heads = repo.remotes["origin"].list_heads(callbacks=BitbucketCallbacks())

In [40]: [head.name for head in heads]
Out[40]: 
['HEAD',
 'refs/heads/1',
 'refs/heads/1.0.0',
 ...,
 'refs/heads/main-3.4.0',
 'refs/heads/nicktest-3.4.0',
 ...]

[beamerblvd]

Pull request: #1465