Recommended directory structure for polyglot projects: SAS and Python

[Quentin]

As I've doing doing more with Python, it raises the question of how to structure a project directory for a project where you use Python and SAS (and not necessarily one called by the other).

I've used a standard SAS directory structure for years, but haven't thought about Python structures, or SAS/Python structure.

AI suggests below, which looks like a reasonable starting point:

my_polyglot_project/
├── .gitignore
├── README.md
├── pyproject.toml             # Python dependencies and project metadata
├── config/                    # Shared configurations
│   ├── db_config.yaml
│   └── sas_config.cfg         # SAS system or SASPy configurations
├── data/                      # Local data directory (do not commit to Git)
│   ├── raw/                   # Immutable raw data (e.g., .csv, .sas7bdat)
│   └── processed/             # Output data from Python/SAS runs
├── notebooks/                 # Jupyter Notebooks for prototyping
├── src/                       # Main Python codebase (Source layout)
│   └── my_project/
│       ├── __init__.py
│       ├── main.py            # Main entry point orchestration script
│       ├── pipeline.py        # Logic coordinating Python and SAS tasks
│       └── utils.py
├── sas/                       # Main SAS codebase
│   ├── macros/                # Reusable SAS macros (.sas files)
│   ├── main_process.sas       # Main entry point SAS script
│   └── scripts/               # Individual SAS ETL or analytical tasks
└── tests/                     # Test suite
    ├── test_python/           # Pytest unit tests
    └── test_sas/              # SAS validation scripts

Looking for other thoughts of directory schemes for polyglot projects.

Thx!

[yabwon]

A few free flying thoughts 🙂

1) You're missing directory for SAS packages 😉

2) I'm not sure if this will be useable in an "enterprise" setup:

sas_config.cfg

rather autoexec.sas?

3) Since SAS has "scripts" and "macros" I'd guess in case of Python you also can have "general use programs" and scripts.

4) I'd also add somethin like "one_off_programs" to separate reusable form not reusable;

5) Why Python's dir is "scr", not "python"? Maybe a top level directory like:  "programs", and then sas and pythons inside it; that would allow for adding other languages;

6) In data maybe it's good idea to create subdirs in "processed", for different kinds of output: datasets, pdfs, htmls, etc.

All the best

Bart

[Quentin]

Thanks Bart.  Yes, this isn't the ideal SAS structure, and I would certainly modify the SAS components (including the director for packages. : )

I'm more curious for thoughts from Python side.  This is basically a very Pythonic structure, with a couple directories added for SAS.  

So when I use poetry to create a new python project directory it makes:

my-project/
├── pyproject.toml    # Project metadata and dependencies
├── README.md         # Project documentation
├── src/
│   └── my_project/   # Main package code directory
│       └── __init__.py
└── tests/            # Unit tests directory
    └── __init__.py

In the joint SAS & Python setting, my inclination would be like yours.  Change /src to /python, and add a /sas directory.  

But what I don't know from the Python side, is whether various python tools (linters, compilers, even VS code) will work better if they can find the standard /src directory, such that renaming it to /python could make my life harder down the road.