Simple Python script that dumps a Wikidot wiki forum into JSON structures
Script is partially based on 2stacks by bluesoul: https://github.com/scuttle/2stacks
To dump a Wiki’s forum, edit config.py and put the required Wiki name (default — scp-wiki, which is the SCP EN community).
If your forum is too large, you may edit threads parameter in config.py, however make sure you don’t overload Wikidot with requests — no one knows what that may result in.
Then make sure you have the following Python packages:
beautifulsoup4
Run python . to start dumping.
Note: incremental dump (i.e. updating existing categories or threads) is not supported.
However, if the process is interrupted at any point, it can be seamlessly resumed per-category and per-thread.
The dump will be stored under the following structure:
dump/
categories.json -- contains group and category names and IDs
categories/
.json -- contains list of threads for each category