Skip to content

Commit f63be3b

Browse files
committed
fixed readme working and layout
1 parent 9cf85b0 commit f63be3b

1 file changed

Lines changed: 22 additions & 25 deletions

File tree

README.rst

Lines changed: 22 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ Using PIP via Github (more likely the latest version)::
2525
2626
*Install note for python newcomers:*
2727

28-
After you've installed pip and python, python scripts are unlikely to be included in your ``$PATH`` by default, this means it cannot be run directly in terminal with ``$ github-backup ...``, you can either add python's install path to your environments ``$PATH`` or call the script directly e.g. using ``$ ~/.local/bin/github-backup``.*
28+
Python scripts are unlikely to be included in your ``$PATH`` by default, this means it cannot be run directly in terminal with ``$ github-backup ...``, you can either add python's install path to your environments ``$PATH`` or call the script directly e.g. using ``$ ~/.local/bin/github-backup``.*
2929

3030
Basic Help
3131
===========
@@ -195,9 +195,9 @@ Github Rate-limit and Throttling
195195

196196
``github-backup`` will automatically throttle itself based on feedback from the Github API.
197197

198-
Their API is usually rate-limited to 5000 calls per hour, and it tells github-backup when to pause and wait until a specific time when the limit is reset.
198+
Their API is usually rate-limited to 5000 calls per hour. The API will ask github-backup to pause until a specific time when the limit is reset again (at the start of the next hour). This continues until the backup is complete.
199199

200-
During a large backup such as ``--all-starred``, and on a fast connection this can result in (~20 min) pauses with bursts of API calls periodically maxing out the API limit. If this is not suitable `it has been observed <https://github.com/josegonzalez/python-github-backup/issues/76#issuecomment-636158717>`_ under real-world conditions that overriding the throttle with ``--throttle-limit 5000 --throttle-pause 0.6`` provides a smooth rate across the hour, although a ``--throttle-pause 0.72`` (3600 seconds [1 hour] / 5000 limit) is theoretically safer to prevent pauses.
200+
During a large backup such as ``--all-starred``, and on a fast connection this can result in (~20 min) pauses with bursts of API calls periodically maxing out the API limit. If this is not suitable `it has been observed <https://github.com/josegonzalez/python-github-backup/issues/76#issuecomment-636158717>`_ under real-world conditions that overriding the throttle with ``--throttle-limit 5000 --throttle-pause 0.6`` provides a smooth rate across the hour, although a ``--throttle-pause 0.72`` (3600 seconds [1 hour] / 5000 limit) is theoretically safer to prevent rate-limit pauses.
201201

202202

203203
About Git LFS
@@ -211,20 +211,20 @@ Instructions on how to do this can be found on https://git-lfs.github.com.
211211
Gotchas / Known-issues
212212
======================
213213

214-
All is not all
215-
--------------
214+
All is not everything
215+
---------------------
216216

217217
The ``--all`` argument does not include; cloning private repos (``-P, --private``), cloning forks (``-F, --fork``) cloning starred repositories (``--all-starred``), ``--pull-details``, cloning LFS repositories (``--lfs``), cloning gists (``--starred-gists``) or cloning starred gist repos (``--starred-gists``). See examples for more.
218218

219219
Cloning all starred size
220220
------------------------
221221

222-
Using the ``--all-starred`` argument to clone all starred repositories may use a large amount of storage space, especially if ``--all`` or more arguments are used. e.g. thousands of JSON issues files, assets and the repos.
222+
Using the ``--all-starred`` argument to clone all starred repositories may use a large amount of storage space, especially if ``--all`` or more arguments are used. e.g. thousands of JSON issues files, assets and the repos etc. Consider just storing the links to starred repos with ``--starred``.
223223

224224
Incremental Backup
225225
-------------------
226226

227-
Incremental (``-i, --incremental``) will request only new data from the API since the last run (successful or not). e.g. only request issues from the API since the last run.
227+
Using (``-i, --incremental``) will request only new data from the API since the last run (successful or not). e.g. only request issues from the API since the last run.
228228

229229
This means any blocking errors on previous runs can cause a large amount of missing data in backups.
230230

@@ -233,43 +233,41 @@ Known blocking errors
233233

234234
Some errors will block the backup by exit the script, such as receiving a 403 Forbidden error from the Github API.
235235

236-
If the incremental argument is used, this will result in the next backup only requesting API data since the last blocked/failed run.
236+
If the incremental argument is used, this will result in the next backup only requesting API data since the last blocked/failed run. Potentially causing unexpected large amounts of missing data.
237237

238-
It's therefore recommended to only use the incremental argument if the output/result is being actively monitored to avoid unexpected missing data in a regular backup runs.
238+
It's therefore recommended to only use the incremental argument if the output/result is being actively monitored, or complimented with periodic full non-incremental runs, to avoid unexpected missing data in a regular backup runs.
239239

240240
1. **Starred public repo blocking**
241241

242242
Since the ``--all`` argument includes ``--hooks``, if you use ``--all`` and ``--all-starred`` together to clone a users starred public repositories, the backup will likely error and block the backup continuing.
243243

244-
This is due to needing the correct permission for ``-hooks`` on public repos.
244+
This is due to needing the correct permission for ``--hooks`` on public repos.
245245

246246
2. **Releases blocking**
247247

248-
A known ``--releases`` (required for ``--assets``) error will sometimes block the backup. If you're backing up a lot of repositories with releases e.g. an organisation or ``--all-starred``.
248+
A known ``--releases`` (required for ``--assets``) error will sometimes block the backup.
249249

250-
You may need to remove ``--releases`` (and therefore ``--assets``) to complete a backup. Documented in `issue 209 <https://github.com/josegonzalez/python-github-backup/issues/209>`_.
250+
If you're backing up a lot of repositories with releases e.g. an organisation or ``--all-starred``. You may need to remove ``--releases`` (and therefore ``--assets``) to complete a backup. Documented in `issue 209 <https://github.com/josegonzalez/python-github-backup/issues/209>`_.
251251

252252

253253
"bare" is actually "mirror"
254254
--------------------------
255255

256-
Using the bare clone argument (``--bare``) will actually call git's ``clone --mirror`` command. There's a subtle difference between `bare <https://www.git-scm.com/docs/git-clone#Documentation/git-clone.txt---bare>`_ and `mirror <https://www.git-scm.com/docs/git-clone#Documentation/git-clone.txt---mirror>`_ clone.::
257-
258-
Compared to --bare, --mirror not only maps local branches of the source to local branches of the target, it maps all refs (including remote-tracking branches, notes etc.) and sets up a refspec configuration such that all these refs are overwritten by a git remote update in the target repository.
256+
Using the bare clone argument (``--bare``) will actually call git's ``clone --mirror`` command. There's a subtle difference between `bare <https://www.git-scm.com/docs/git-clone#Documentation/git-clone.txt---bare>`_ and `mirror <https://www.git-scm.com/docs/git-clone#Documentation/git-clone.txt---mirror>`_ clone. ::
257+
258+
Compared to --bare, --mirror not only maps local branches of the source to local branches of the target, it maps all refs (including remote-tracking branches, notes etc.) and sets up a refspec configuration such that all these refs are overwritten by a git remote update in the target repository.
259259

260260

261-
Starred gists stored with user gists
262-
------------------------------------
261+
Starred gists vs starred repo behaviour
262+
---------------------------------------
263263

264-
The starred repo cloning (``--all-starred``) argument stores starred repos separately to the users own repositories. However, using ``--starred-gists`` will store starred gists within the same directory as the users own gists ``--gists``.
264+
The starred normal repo cloning (``--all-starred``) argument stores starred repos separately to the users own repositories. However, using ``--starred-gists`` will store starred gists within the same directory as the users own gists ``--gists``. Also, all gist repo directory names are IDs not the gist's name.
265265

266266

267267
Skip existing on incomplete backups
268268
-------------------------------------------------------
269269

270-
The ``--skip-existing`` argument will skip a backup if the directory already exists, regardless of if the backup in that directory was not successfully completed (perhaps due to a blocking error).
271-
272-
This may result in unexpected missing data in a regular backup.
270+
The ``--skip-existing`` argument will skip a backup if the directory already exists, regardless of if the backup in that directory was not successfully completed (perhaps due to a blocking error). This may result in unexpected missing data in a regular backup.
273271

274272

275273
Github Backup Examples
@@ -288,14 +286,14 @@ Use a fine-grained access token to backup a single organization repository with
288286
# e.g. git@github.com:docker/cli.git
289287
github-backup $ORGANIZATION -P -f $FINE_ACCESS_TOKEN -o . --all -O -R $REPO
290288

291-
Quietly and incrementally backup useful Github user data (public and private repos with SSH) including; all issues, pulls, all public starred repos and gists (omitting "hooks", "releases" and therefore "assets" to prevent blocking). *Great for a cron job.*::
289+
Quietly and incrementally backup useful Github user data (public and private repos with SSH) including; all issues, pulls, all public starred repos and gists (omitting "hooks", "releases" and therefore "assets" to prevent blocking). *Great for a cron job.* ::
292290

293291
export FINE_ACCESS_TOKEN=SOME-GITHUB-TOKEN
294292
GH_USER=YOUR-GITHUB-USER
295293

296294
github-backup -f $FINE_ACCESS_TOKEN --prefer-ssh -o ~/github-backup/ -l error -P -i --all-starred --starred --watched --followers --following --issues --issue-comments --issue-events --pulls --pull-comments --pull-commits --labels --milestones --repositories --wikis --releases --assets --pull-details --gists --starred-gists $GH_USER
297295
298-
Debug an erroring/blocking or incomplete backup into a temporary directory. Omit "incremental" to fix a previous incomplete backup.::
296+
Debug an erroring/blocking or incomplete backup into a temporary directory. Omit "incremental" to fix a previous incomplete backup. ::
299297

300298
export FINE_ACCESS_TOKEN=SOME-GITHUB-TOKEN
301299
GH_USER=YOUR-GITHUB-USER
@@ -304,11 +302,10 @@ Debug an erroring/blocking or incomplete backup into a temporary directory. Omit
304302

305303

306304

307-
308305
Development
309306
===========
310307

311-
This project is considered feature complete for the primary maintainer. If you would like a bugfix or enhancement and can not sponsor the work, pull requests are welcome. Feel free to contact the maintainer for consulting estimates if desired.
308+
This project is considered feature complete for the primary maintainer @josegonzalez. If you would like a bugfix or enhancement, pull requests are welcome. Feel free to contact the maintainer for consulting estimates if you'd like to sponsor the work instead.
312309

313310
Testing
314311
-------

0 commit comments

Comments
 (0)