Add support for the Atom tool

README.rst

@@ -1,107 +1,49 @@
-A Simple Python Project Skeleton
-================================
+=================
+ReachableCode
+=================
 
-This repo attempts to standardize the structure of the Python-based project's
-repositories using modern Python packaging and configuration techniques.
-Using this `blog post`_ as inspiration, this repository serves as the base for
-all new Python projects and is mergeable in existing repositories as well.
+ReachableCode is a collection of pipelines for analyzing code reachability.
+It integrates with ScanCode.io as a pipeline plugin
+and uses tools like (Atom, CPG, .. ) to determine whether
+vulnerable code symbols are reachable within a project's codebase.
 
-.. _blog post: https://blog.jaraco.com/a-project-skeleton-for-python-projects/
+Other information:
 
+- License: Apache-2.0
+- Copyright (c) nexB Inc., AboutCode.org and others.
+- Homepage: https://github.com/aboutcode-org/reachablecode/
++ Documentation: https://reachablecode.readthedocs.io/
 
-Usage
-=====
+Requirements
+------------
 
-A brand new project
--------------------
+- ScanCode.io https://github.com/nexB/scancode.io
 
-.. code-block:: bash
+See also the next section for detailed instructions.
 
-    git init my-new-repo
-    cd my-new-repo
-    git pull git@github.com:nexB/skeleton
 
-    # Create the new repo on GitHub, then update your remote
-    git remote set-url origin git@github.com:nexB/your-new-repo.git
+Local installation and usage test
+------------------------------------
 
-From here, you can make the appropriate changes to the files for your specific project.
+To install:
+~~~~~~~~~~~~
 
-Update an existing project
----------------------------
+- Clone reachablecode locally side-by-side: ``git clone https://github.com/aboutcode-org/reachablecode``
+- Clone ScanCode.io locally side-by-side: ``git clone https://github.com/aboutcode-org/scancode.io``
+- Change to the scancode.io directory and run ``make dev`` then ``source bin/activate``
 
-.. code-block:: bash
+  - Follow the full instructions at https://scancodeio.readthedocs.io/en/latest/installation.html#local-development-installation
 
-    cd my-existing-project
-    git remote add skeleton git@github.com:nexB/skeleton
-    git fetch skeleton
-    git merge skeleton/main --allow-unrelated-histories
+- Run ``pip install --editable ../reachablecode``
+- Run ScanCode.io with ``./manage.py runserver --insecure`` and open the URL in your browser.
+  There is a new "analyze_atom_reachability" pipeline available when creating a new project.
 
-This is also the workflow to use when updating the skeleton files in any given repository.
+Development
+----------------
 
-More usage instructions can be found in ``docs/skeleton-usage.rst``.
+- Install requirements and dependencies using ``make dev``
+- Then ``source venv/bin/activate``
 
+Testing:
 
-Release Notes
-=============
-
-- 2025-03-31:
-
-    - Use ruff as the main code formatting tool, add ruff rules to pyproject.toml
-
-- 2025-03-29:
-
-    - Add support for beta macOS-15
-    - Add support for beta windows-2025
-
-- 2025-02-14:
-
-    - Drop support for Python 3.8, add support in CI for Python 3.13, use Python 3.12 as default
-      version.
-
-- 2025-01-17:
-
-    - Drop support for macOS-12, add support for macOS-14
-    - Add support in CI for ubuntu-24.04
-    - Add support in CI for Python 3.12
-
-- 2024-08-20:
-
-    - Update references of ownership from nexB to aboutcode-org
-
-- 2024-07-01:
-
-    - Drop support for Python 3.8
-    - Drop support for macOS-11, add support for macOS-14
-
-- 2024-02-19:
-
-    - Replace support in CI of default ubuntu-20.04 by ubuntu-22.04
-
-- 2023-10-18:
-
-    - Add dark mode support in documentation
-
-- 2023-07-18:
-
-    - Add macOS-13 job in azure-pipelines.yml
-
-- 2022-03-04:
-
-    - Synchronize configure and configure.bat scripts for sanity
-    - Update CI operating system support with latest Azure OS images
-    - Streamline utility scripts in etc/scripts/ to create, fetch and manage third-party
-      dependencies. There are now fewer scripts. See etc/scripts/README.rst for details
-
-- 2021-09-03:
-    - ``configure`` now requires pinned dependencies via the use of ``requirements.txt``
-      and ``requirements-dev.txt``
-    - ``configure`` can now accept multiple options at once
-    - Add utility scripts from scancode-toolkit/etc/release/ for use in generating project files
-    - Rename virtual environment directory from ``tmp`` to ``venv``
-    - Update README.rst with instructions for generating ``requirements.txt``
-      and ``requirements-dev.txt``, as well as collecting dependencies as wheels and generating
-      ABOUT files for them.
-
-- 2021-05-11:
-    - Adopt new configure scripts from ScanCode TK that allows correct configuration of which
-      Python version is used.
+- To run tests: ``pytest -vvs``

atom.Dockerfile

@@ -0,0 +1,43 @@
+# SPDX-License-Identifier: Apache-2.0
+#
+# http://nexb.com and https://github.com/aboutcode-org/scancode.io
+# The ScanCode.io software is licensed under the Apache License version 2.0.
+# Data generated with ScanCode.io is provided as-is without warranties.
+# ScanCode is a trademark of nexB Inc.
+#
+# You may not use this software except in compliance with the License.
+# You may obtain a copy of the License at: http://apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software distributed
+# under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
+# CONDITIONS OF ANY KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations under the License.
+#
+# Data Generated with ScanCode.io is provided on an "AS IS" BASIS, WITHOUT WARRANTIES
+# OR CONDITIONS OF ANY KIND, either express or implied. No content created from
+# ScanCode.io should be considered or used as legal advice. Consult an Attorney
+# for any legal advice.
+#
+# ScanCode.io is a free software code scanning tool from nexB Inc. and others.
+# Visit https://github.com/aboutcode-org/scancode.io for support and download.
+
+FROM ghcr.io/appthreat/atom:v3.2.0
+
+RUN set -e; \
+    JAVA_DIR="$(ls -d /opt/java/*-tem | head -n1)"; \
+    MAVEN_DIR="$(ls -d /opt/maven/* | head -n1)"; \
+    SBT_DIR="$(ls -d /opt/sbt/* | head -n1)"; \
+    SCALA_DIR="$(ls -d /opt/scala/* | head -n1)"; \
+    mkdir -p /atom-staged/opt /atom-staged/bin /atom-staged/lib/node_modules; \
+    cp -a /opt/wrapper /opt/java /opt/maven /opt/sbt /opt/scala /opt/vendor /atom-staged/opt/; \
+    cp /usr/bin/node-22 /atom-staged/bin/node; \
+    for lib in $(ldd /usr/bin/node-22 | awk '/=>/{print $3}'); do \
+      case "$lib" in */libc.so*|*/libm.so*|*/libpthread*|*/libdl.so*|*/librt*|*/ld-linux*) ;; *) cp -aL "$lib" /atom-staged/lib/ ;; esac; \
+    done; \
+    cp -a /usr/local/lib/node_modules/. /atom-staged/lib/node_modules/; \
+    ln -sfn /atom-root/opt/wrapper/nodejs/packages/atom /atom-root/opt/wrapper/nodejs/packages/atom-jar /atom-staged/lib/node_modules/@appthreat/; \
+    printf '#!/bin/sh\nexport LD_LIBRARY_PATH=/atom-root/lib\nexport JAVA_HOME=/atom-root%s\nexport PATH=/atom-root%s/bin:/atom-root%s/bin:/atom-root%s/bin:/atom-root%s/bin:/atom-root/opt/vendor/bin:$PATH\nexec /atom-root/bin/node /atom-root/opt/wrapper/nodejs/packages/atom/index.js "$@"\n' \
+      "$JAVA_DIR" "$JAVA_DIR" "$MAVEN_DIR" "$SBT_DIR" "$SCALA_DIR" \
+      > /atom-staged/bin/atom; \
+    chmod +x /atom-staged/bin/atom
+
+ENTRYPOINT ["/bin/sh", "-c", "set -e; find /atom-root -mindepth 1 -delete; cp -a /atom-staged/. /atom-root/"]

configure

@@ -28,14 +28,15 @@ CLI_ARGS=$1
 ################################
 
 # Requirement arguments passed to pip and used by default or with --dev.
-REQUIREMENTS="--editable . --constraint requirements.txt"
-DEV_REQUIREMENTS="--editable .[dev] --constraint requirements.txt --constraint requirements-dev.txt"
+REQUIREMENTS="--editable ."
+DEV_REQUIREMENTS="--editable .[testing,docs]"
+DOCS_REQUIREMENTS="--editable .[docs]"
 
 # where we create a virtualenv
 VIRTUALENV_DIR=venv
 
 # Cleanable files and directories to delete with the --clean option
-CLEANABLE="build dist venv .cache .eggs *.egg-info docs/_build/ pip-selfcheck.json"
+CLEANABLE="build dist venv .cache .eggs"
 
 # extra  arguments passed to pip
 PIP_EXTRA_ARGS=" "
@@ -110,7 +111,7 @@ create_virtualenv() {
         fi
 
         $PYTHON_EXECUTABLE "$VIRTUALENV_PYZ" \
-            --pip embed --setuptools embed \
+            --wheel embed --pip embed --setuptools embed \
             --seeder pip \
             --never-download \
             --no-periodic-update \
@@ -167,7 +168,6 @@ clean() {
     for cln in $CLEANABLE;
         do rm -rf "${CFG_ROOT_DIR:?}/${cln:?}";
     done
-    find . -type f -name '*.py[co]' -delete -o -type d -name __pycache__ -delete
     set +e
     exit
 }
@@ -185,6 +185,7 @@ while getopts :-: optchar; do
                 help  ) cli_help;;
                 clean ) find_python && clean;;
                 dev   ) CFG_REQUIREMENTS="$DEV_REQUIREMENTS";;
+                docs   ) CFG_REQUIREMENTS="$DOCS_REQUIREMENTS";;
             esac;;
     esac
 done

docker-compose.yml

@@ -0,0 +1,131 @@
+name: scancodeio
+
+x-description: "Full ScanCode.io stack: all services, local image build"
+
+services:
+  db:
+    image: docker.io/library/postgres:17.11
+    env_file:
+      - docker.env
+    volumes:
+      - db_data:/var/lib/postgresql/data/
+    shm_size: "1gb"
+    restart: always
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+
+  redis:
+    image: docker.io/library/redis:8.10.2-alpine
+    # Enable redis data persistence using the "Append Only File" with the
+    # default policy of fsync every second. See https://redis.io/topics/persistence
+    command: redis-server --appendonly yes --loglevel warning
+    volumes:
+      - redis_data:/data
+    restart: always
+    healthcheck:
+      test: ["CMD", "redis-cli", "ping"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+
+  web:
+    build:
+      context: ../scancode.io
+    command: sh -c "
+        ./manage.py migrate &&
+        ./manage.py collectstatic --no-input --verbosity 0 --clear &&
+        gunicorn scancodeio.wsgi:application --bind :8000 --timeout 600 \
+          --workers ${GUNICORN_WORKERS:-8} --worker-tmp-dir /dev/shm"
+    env_file:
+      - docker.env
+    expose:
+      - 8000
+    volumes:
+      - .env:/opt/scancodeio/.env
+      - /etc/scancodeio/:/etc/scancodeio/
+      - workspace:/var/scancodeio/workspace/
+      - static:/var/scancodeio/static/
+      - healthycode_venv:/opt/healthycode
+      - atom_root:/atom-root
+    restart: always
+    depends_on:
+      db:
+        condition: service_healthy
+      redis:
+        condition: service_healthy
+
+  worker:
+    build:
+      context: ../scancode.io
+    # Ensure that potential db migrations run first by waiting until "web" is up
+    command: wait-for-it --strict --timeout=600 web:8000 -- sh -c "
+        ./manage.py rqworker --worker-class scancodeio.worker.ScanCodeIOWorker
+                             --queue-class scancodeio.worker.ScanCodeIOQueue
+                             --verbosity 1"
+    env_file:
+      - docker.env
+    volumes:
+      - .env:/opt/scancodeio/.env
+      - /etc/scancodeio/:/etc/scancodeio/
+      - workspace:/var/scancodeio/workspace/
+      - healthycode_venv:/opt/healthycode
+      - atom_root:/atom-root
+    restart: always
+    depends_on:
+      db:
+        condition: service_healthy
+      redis:
+        condition: service_healthy
+      web:
+        condition: service_started
+
+  nginx:
+    image: docker.io/library/nginx:1.31.6-alpine
+    ports:
+      - "${NGINX_PUBLISHED_HTTP_PORT:-80}:80"
+      - "${NGINX_PUBLISHED_HTTPS_PORT:-443}:443"
+    volumes:
+      - ./etc/nginx/conf.d/:/etc/nginx/conf.d/
+      - static:/var/scancodeio/static/
+      - webroot:/var/www/html/
+    depends_on:
+      web:
+        condition: service_started
+    restart: always
+
+  clamav:
+    image: docker.io/clamav/clamav:1.5.4_base
+    volumes:
+      - clamav_data:/var/lib/clamav
+      - workspace:/var/scancodeio/workspace/
+    restart: always
+
+  healthycode:
+    profiles:
+      - healthycode
+    image: ghcr.io/aboutcode-org/healthycode:0.2.0
+    volumes:
+      - healthycode_venv:/opt/healthycode
+    restart: "no"
+
+  atom:
+    profiles:
+      - atom
+    build:
+      context: .
+      dockerfile: atom.Dockerfile
+    volumes: [atom_root:/atom-root]
+    restart: "no"
+
+volumes:
+  db_data:
+  redis_data:
+  clamav_data:
+  static:
+  workspace:
+  webroot:
+  healthycode_venv:
+  atom_root:

docker.env

@@ -0,0 +1,10 @@
+POSTGRES_DB=scancodeio
+POSTGRES_USER=scancodeio
+POSTGRES_PASSWORD=scancodeio
+POSTGRES_INITDB_ARGS=--encoding=UTF-8 --lc-collate=en_US.UTF-8 --lc-ctype=en_US.UTF-8
+
+SCANCODEIO_DB_HOST=db
+SCANCODEIO_RQ_REDIS_HOST=redis
+SCANCODEIO_ASYNC=True
+SCANCODEIO_WORKSPACE_LOCATION=/var/scancodeio/workspace/
+ATOM_EXECUTABLE=/atom-root/bin/atom

docs/source/index.rst

@@ -5,7 +5,7 @@ Welcome to nexb-skeleton's documentation!
    :maxdepth: 2
    :caption: Contents:
 
-   skeleton-usage
+   tutorial_atom_reachability_integration_tools
    contribute/contrib_doc
 
 Indices and tables