wp-parsely / docs / TESTING.md

TESTING.md in Parse.ly 3.16.3, at docs/TESTING.md

148 lines 6.4 KB

Raw Download Zip

1	# Testing
2
3	Besides the commands in this document, additional testing commands can be found within [](../composer.jsoncomposer.json](../composer.json](../composer.json) under the `scripts` and `scripts-descriptions` sections.
4
5	## PHP unit tests
6
7	```
8	# Run all single-site unit tests.
9	composer test
10
11	# Run all multisite unit tests.
12	composer test-ms
13	```
14
15	## PHP integration tests
16
17	### Installing requirements
18
19	In order to run integration tests, an installation script must be executed beforehand. For this to succeed, you will need to:
20
21	- Have the `svn` command installed into your system. If you don't have it, you can install subversion to get it (`brew install subversion` if you're using brew).
22	- Create a new dedicated test database in your local MySQL installation. Here we will assume that the database is named `wp_tests` with a username of `root` and an empty (`""`) password.
23	- If you're on Windows, please also refer to [](WINDOWS.mdWINDOWS.md](WINDOWS.md](WINDOWS.md).
24
25	To run the installation script:
26
27	1. Navigate to the main plugin directory.
28	2. Run the script:
29
30	```
31	./bin/install-wp-tests.sh wp_tests root "" localhost latest true
32	```
33
34	If you are using different credentials/settings, the structure of the command is as follows:
35
36	```
37	./bin/install-wp-tests.sh <db-name> <db-user> <db-pass> [db-host] [wp-version] [skip-database-creation]"
38	```
39
40	Important Notes:
41	- It is recommended to always use `latest` as the value for the `wp-version` argument.
42	- The installation takes place into a temporary directory, which means that the related files will get deleted between reboots. The way to remediate this is just to re-run the installation script whenever needed.
43	- The database will be completely overwritten every time integration tests are executed.
44
45	### Running integration tests
46
47	```
48	# Run all single-site integration tests.
49	composer testwp
50
51	# You can use double dashes to add PHPUnit parameters.
52	composer testwp -- --filter SettingsPageTest
53	# The above will only run the SettingsPage test.
54
55	# Run all multisite integration tests.
56	composer testwp-ms
57	```
58
59	### Troubleshooting
60	- If you encounter any `require` (class not found) issues, you can usually fix them by running `composer dump-autoload`.
61	- If you're getting an error like `ERROR: The WordPress native unit test bootstrap file could not be found. Please set either the WP_TESTS_DIR or the WP_DEVELOP_DIR environment variable, either in your OS or in a custom phpunit.xml file.`, then most probably you need to re-run the integration tests installation script, as the related files get regularly deleted.
62
63	## PHP code coverage
64
65	### Installing requirements
66
67	To be able to run coverage commands, you'll need to install `pcov`:
68
69	```
70	pecl install pcov
71	```
72
73	Some times, `pcov` does not work out of the box if `PHP` has been installed using Homebrew. Here are some possible troubleshooting steps (version strings should be updated to the ones you're using).
74
75	```
76	# If missing, create a symlink to pcre2.h.
77	ln -s /opt/homebrew/Cellar/pcre2/10.42/include/pcre2.h /opt/homebrew/Cellar/php@7.4/7.4.33_1/include/php/ext/pcre/pcre2.h
78
79	# If the /opt/homebrew/Cellar/php@7.4/7.4.33_1/pecl symlink points to a
80	# directory that does not exist, create the directory.
81	mkdir /opt/homebrew/lib/php/pecl
82	```
83
84	### Running code coverage
85
86	```
87	composer coverage
88	```
89
90	## JavaScript tests
91
92	```
93	# Run front-end tests.
94	npm run test
95	```
96
97	## End-to-end (E2E) tests
98
99	This suite is meant to simulate actual user actions as they interact with the plugin. Tests are run against a real WordPress instance and activities are performed in a real browser. The idea is that we can provide confidence that changes going forward have the intended effect on the DOM and rendered content that plugin users and site visitors will see under various specific conditions.
100
101	### How E2E tests work
102
103	In order for the tests to do their job, they need a WordPress instance. To that end, we spin up a bare-bones containerized site and configure it to the default values for the WordPress `e2e-tests` helper. We then leverage the `@wordpress/scripts` utility's [](https://developer.wordpress.org/block-editor/reference-guides/packages/packages-scripts/#test-e2ebuilt-in functionality](https://developer.wordpress.org/block-editor/reference-guides/packages/packages-scripts/#test-e2e](https://developer.wordpress.org/block-editor/reference-guides/packages/packages-scripts/#test-e2e) to launch a browser via [](https://pptr.dev/Puppeteer](https://pptr.dev/](https://pptr.dev/).
104
105	The tests use the [](https://jestjs.io/Jest framework](https://jestjs.io/](https://jestjs.io/) to drive a user flow and assert on expected outcomes. In addition to the [](https://github.com/puppeteer/puppeteer/blob/main/docs/api.mdPuppeteer API](https://github.com/puppeteer/puppeteer/blob/main/docs/api.md](https://github.com/puppeteer/puppeteer/blob/main/docs/api.md), there are a number of helpers to accomplish frequently performed tasks in the [](https://developer.wordpress.org/block-editor/reference-guides/packages/packages-e2e-test-utils/`@wordpress/e2e-test-utils` package](https://developer.wordpress.org/block-editor/reference-guides/packages/packages-e2e-test-utils/](https://developer.wordpress.org/block-editor/reference-guides/packages/packages-e2e-test-utils/).
106
107	See this post for more information: https://make.wordpress.org/core/2019/06/27/introducing-the-wordpress-e2e-tests/
108
109	### How to run E2E tests
110
111	- Make sure that [](CONTRIBUTING.md#minimum-requirementsrequirements](CONTRIBUTING.md#minimum-requirements](CONTRIBUTING.md#minimum-requirements) are installed and that you have the dependencies up to date by running `npm install`.
112
113	- Start the back-end:
114
115	- From the `wp-parsely` directory, run `npm run dev:start`. Once you see a line that says `✔ Done! (in XXXs YYYms)`, you may proceed.
116
117	- Run the tests:
118
119	- Once your environment is ready, you can launch the e2e tests suite by running `npm run test:e2e`. This will run the test suite using a headless browser.
120
121	- For debugging purpose, you might want to follow the test visually. You can do so by running the tests in an interactive mode: `npm run test:e2e:interactive`
122
123	- You can also run a given test file separately: `npm run test:e2e tests/e2e/specs/activation-flow.spec.js`
124
125	- Finish:
126
127	- When you're finished testing, the back-end containers and storage can be stopped using `npm run dev:stop`.
128
129	### Automated E2E testing
130
131	Our E2E tests are hooked into a GitHub workflow called [](../.github/workflows/e2e-tests.ymlEnd-to-end (e2e) Tests](../.github/workflows/e2e-tests.yml](../.github/workflows/e2e-tests.yml), which uses the same environment mentioned above to spin-up a WordPress environment. Commits pushed to GitHub will be tested against that environment using our E2E tests.
132
133	## Manual smoke test
134
135	Before releasing a new wp-parsely version, we should perform a basic manual smoke test which includes the following actions:
136
137	- Disable and re-enable the plugin
138	- Visit the settings page and click on the tabs
139	- Update a setting using the settings page
140	- Try to save an invalid Site ID and API Secret combination
141	- Try the Content Helper Widget and Sidebar with:
142	- An empty API Secret
143	- No Site ID
144	- Check that The Content Helper Post List Stats appears
145	- Check that "ld+json" and "parsely-cfg" are injected into the front-end
146
147	This list is currently a work in progress.
148