Generate sphinx docu from docstrings not working

Question

I have a project with the following structure (which I would like to keep):

my_project
├── build  # here is where sphinx should dump into
├── requirements.txt
├── make.bat
├── Makefile
├── ...  # more config files
├── doc  # this is where I want sphinx files to live
│   ├── conf.py
│   └── index.rst
├── src
│   └── my_project
│       ├── __init__.py
│       ├── module_1
│       │   ├── __init__.py
│       │   └── ...
│       └── util
│           ├── __init__.py
│           └── ...
└── tests
    ├── module_1
    │   ├── __init__.py
    │   └── ...  # testing module 1
    └── util
        ├── __init__.py
        └── ...  # testing util stuff

I recreated it on github, which can be used to recreate the results by executing my_setup.sh within.

I want to build the documentation from docstrings. I used sphinx's quickstart to generate necessary config, but when I call make hmtl, the resulting docu doesn't include any docstrings from my source code, i.e. everything in my_project/src/my_project. Sphinx's documenation is a bit overwhelming, given that I feel that I am trying to set up something very basic.

Relevant info from config files (please tell me if I forgot something important):

Makefile

SPHINXOPTS    =
SPHINXBUILD   = sphinx-build
SPHINXPROJ    = my_project
SOURCEDIR     = doc
BUILDDIR      = build
...

make.bat

set SOURCEDIR=doc
set BUILDDIR=build
set SPHINXPROJ=my_project
...

conf.py

import os
import sys
sys.path.insert(0, os.path.abspath('../src/my_project'))
...
extensions = [
    'sphinx.ext.autodoc',
    'sphinx.ext.todo',
    'sphinx.ext.coverage',
]
...

I tried this as well, but it first put a bunch of build files into doc that I'd rather not have there and it also didn't find any of the modules (fixed by omitting the -F parameter):

$ sphinx-apidoc -F -o doc/ src/my_project/
$ cd doc
$ make html
Running Sphinx v1.7.2
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 0 source files that are out of date
updating environment: 0 added, 2 changed, 0 removed
reading sources... [100%] my_project.util                                                                                                                                                                                  
WARNING: autodoc: failed to import module 'my_project'; the following exception was raised:
No module named 'my_project'
WARNING: autodoc: failed to import module 'my_project.util.test_file'; the following exception was raised:
No module named 'my_project'
WARNING: autodoc: failed to import module 'my_project.util'; the following exception was raised:
No module named 'my_project'
looking for now-outdated files... none found
pickling environment... done
checking consistency... /home/arne/workspace/git/my_project/doc/my_project.rst: WARNING: document isn\'t included in any toctree
done
preparing documents... done
writing output... [100%] my_project.util                                                                                                                                                                                   
generating indices... genindex
writing additional pages... search
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded, 4 warnings.

The problem might be that your original Sphinx project that was created with sphinx-quickstart gets overwritten. When you use the `-F` option, sphinx-apidoc creates a new Sphinx project (with a different conf.py, Makefile etc.). — mzjn, Apr 17 '18 at 07:31
@mzjn That was definitely a problem, using `sphinx-apidoc -o build src/my_project` instead put the files in the right place and works with the original makefile. But the resulting page still won't include any docstring info. — Arne, Apr 17 '18 at 07:40
Somewhere you need to have one `automodule` directive for each module. You can add them by hand in index.rst (for example) or let sphinx-apidoc create additional rst files with these directives. What do your `automodule` directives look like? — mzjn, Apr 17 '18 at 07:43
[`my_module.rst`](http://dpaste.com/10J31XT), [`my_module.util.rst`](http://dpaste.com/195CG9E). There is also [`modules.rst`](http://dpaste.com/140S9QY) in there, but it doesn't have any automodule info. — Arne, Apr 17 '18 at 07:50
This is confusing. In your project structure diagram there is nothing called `my_module`. It says that you have a package called `my_project` with `module_1` and `util` submodules. — mzjn, Apr 17 '18 at 07:56
my bad, my project isn't actually called `my_project`.. and I copied the wrong stand-in over. I think every my_module needs to be called my_project instead, and module_1's rst would look [like this](http://dpaste.com/2W1KRM6). — Arne, Apr 17 '18 at 08:10
I think you need to use `sys.path.insert(0, os.path.abspath('../src/'))`. — mzjn, Apr 17 '18 at 08:16
Still nothing.. just in case, [this](http://dpaste.com/3FVHR3N) is the test file I created that I hope to extract the dosctrings from. — Arne, Apr 17 '18 at 08:30
Minor comment, I think you mean `my_setup.sh`, and I would suggest using the first line as follows `python3 -m venv env`. Also toss in a `pip install --upgrade pip`. I'll follow up with more information if I can figure out what's going on. — Steve Piercy, Apr 18 '18 at 06:10
@Arne using the `my_project\SRC` layout, I found the explanation in [this thread invaluable](https://stackoverflow.com/questions/36579500/can-i-remove-the-src-package-from-python-doc-generated-with-sphinx) as to why `SRC` is magically omitted from package names. As a complementing afterthought... — bad_coder, Apr 27 '20 at 05:13
@bad_coder It's been a while since I asked this question, by now I changed my default project layout to one where the package is dev-installed before sphinx starts to build docs. That way, I don't need to handle the PYTHONPATH myself, which solved a whole class of problems for me. [This post](https://stackoverflow.com/questions/50155464/using-pytest-with-a-src-layer) explained me the pros and cons in that regard. — Arne, Apr 27 '20 at 06:12

score 5 · Accepted Answer · answered Apr 18 '18 at 06:27

There are a couple of issues with your MCVE.

rST source files must not reside in the output directory build, and should be in the docs source directory docs. You should have done this instead: sphinx-apidoc -o docs src/my_project.

As @mzjn mentioned, you need to uncomment and add some lines to your conf.py to resolve the WARNING: autodoc: failed to import module errors.

# -- Path setup --------------------------------------------------------------

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
import os
import sys
# sys.path.insert(0, os.path.abspath('.'))
sys.path.insert(0, os.path.abspath('../src/'))

After those two changes, I was able to successfully build your docs with its API.

Generate sphinx docu from docstrings not working

1 Answers1

Linked