Page Generation

Wellwiki uses four primary scripts in order to generate new pages. All of these scripts are located within ~/installers/wikimanager/

pre_deployment.sql

The first script updates the violation counts for wells in the well table of wellsdb_dev. It can be run from mysql by logging in as root, selecting wellsdb_dev as the database of choice, and running source ~/installers/wikimanager/pre_deployment.sql within a mysql console.

The script takes about 1 minute to fully execute.

well_updates.py

This script updates the well table based on information in permit, production, spud, municipality, and county. It should be run after new data is added to the database to ensure that well info is fully populated. It is located in ~/scripts/ and is run using python well_updates.py in this directory. If only specific update functions need to be carried out, this can be specified in if __name__=='__main__':.

UpdateManagerPageTable.py

This is a python2 script which adds any wells within the mysql wellsdb_dev database to the wikiManagerPages table within the wikidb database. This should be run prior to CreateEntitySources such that all necessary pages needed to be generated are able to be flagged for updates.

Flagging Operators for Update

In addition to flagging wells for update, operator pages should also be set to be updated. This can be done on a state by state basis with the following:

UPDATE wikidb.wikiManagerPages

SET needs_update='1'

WHERE page_name IN

(SELECT page_name FROM

(SELECT u1.* FROM wikidb.wikiManagerPages u1 LEFT JOIN wellsdb_dev.operator u2 ON u1.page_name=u2.operator_name WHERE u2.operator_state='PA') AS p);

CreateEntitySources.py

This is a python2 script that will read through all wiki pages located in the wikiManagerPages table within the wikidb database that have "NEEDS_UPDATE" set to 1.

It uses scripts within DBUtil/ to query the mysql database, which in turn return dictionaries for each well in question in which keys correspond with column names from the mysql database, and values are the corresponding entries for a given well. Scripts within FileUtil/SourceBuilders/ generate page templates and output parsed files using well specific data from DBUtil results. Note that the *SourceBuilder.py scripts are used, while the *TableTemplateBuilder.py scripts are older versions of these and are therefore unused.

Each entry within the wikiManagerPages table will be queried for in the wellsdb_dev database, which then outputs newly generated pages to ~/PycharmProjects/WikiManager/WikiSources

Here is some sample SQL to flag all Pennsylvania pages for update:

UPDATE `wikiManagerPages`

SET `NEEDS_UPDATE`='1'

WHERE `PAGE_NAME` LIKE '37%'

This script is currently fully functional for at least the state of Pennsylvania.

commit_staged_sources.py

This python2 file takes a source directory as an argument, and begins adding all wiki pages within that source directory to the actual wikimedia database, making them available for viewing here.

Usage: python commit_staged_sources.py ~/PycharmProjects/WikiManager/WikiSources

add_county_info_boxes.py

This python2 file adds Wikipedia infoboxes to county pages. It should be run exactly one time after county pages have been flagged for update and CreateEntitySources.py and commit_staged_sources.py have executed. Inboxes are added to all county pages that exist in both wikiPageUpload and wikiManagerPages.

---

BC Well Pages

Currently, all BC well pages are generated at once (not selectively by flagging individual pages).

CreateEntitySources.py

Uncomment the lines dbClient = DBClient() (57) and CreateBCSources(dbClient) (60) and run as python CreateEntitySources. Newly generated pages are output to ~/PycharmProjects/WikiManager/WikiSources. Re-comment these lines when finished so that BC pages are not generated every time non-BC pages are generated.

commit_staged_sources.py

This file adds all wiki pages within a source directory to the actual wikimedia database, making them available for viewing on the website.

Usage: python commit_staged_sources.py ~/PycharmProjects/WikiManager/WikiSources

EntityController.py

The selection of data from the wellwiki_dev database occurs here. If it is desired to update only pages matching specific criteria, a WHERE clause can be manually appended to sqlStr.

---

AB Well Pages

Currently, all AB well and operator pages are generated at once (not selectively by flagging individual pages).

pre_deployment_AB.sql

The script updates the violation counts for wells in the ab_well table of wellsdb_dev. It can be run from mysql by logging in as root, selecting wellsdb_dev as the database of choice, and running source ~/installers/wikimanager/pre_deployment_AB.sql; within a mysql console.

CreateEntitySources.py

Uncomment the lines dbClient = DBClient() (57) and CreateABSources(dbClient) (59) and run as python CreateEntitySources. Newly generated pages are output to ~/PycharmProjects/WikiManager/WikiSources. Re-comment these lines when finished so that AB pages are not generated every time non-AB pages are generated.

create_orphan_ab_page.py

Running this python2 file causes the AB Orphan Wells page source text file to be generated in ~/PycharmProjects/WikiManager/WikiSources.

commit_staged_sources.py

see above

EntityController.py

The selection of data from the wellwiki_dev database occurs here. If it is desired to update only pages matching specific criteria, a WHERE clause can be manually appended to sqlStr in GetAllWells and/or GetAllOperators.