In short: Pentest or vulnerability scan is not an either-or question but a question of order. A vulnerability scan finds known vulnerabilities and misconfigurations automatically, quickly and cheaply, but with false alarms and blind spots. A penetration test examines specifically what an attacker could actually achieve. A bug bounty programme opens the testing to external researchers and pays off only when scans and pentests are routine. For most SMEs the order is: scan regularly, fix the findings, have targeted tests carried out, set up a reporting point for vulnerabilities.
Why the question is being asked anew after PHP2Shell
Anyone who tests their website with an online scanner after PHP2Shell may get confusing results. One scanner may report a critical vulnerability although the host has confirmed the patch. Another reports nothing although the server still uses an old PHP version, because the version information is hidden.
The reason lies in how they work. Many scanners infer vulnerability from the displayed version. Distributions, however, install backports without changing the version number; according to reports, Ubuntu 24.04, for example, continues to show 8.3.6 after the update. Why this is so is explained in Why php -v shows the same number after the update. Reliable evidence comes from the package changelog, see Checking the PHP version and patch level correctly.
This is not an argument against scanners but one for knowing their limits and combining them with other methods.
Pentest or vulnerability scan: the three methods compared
| Criterion | Vulnerability scan | Penetration test | Bug bounty |
|---|---|---|---|
| Who tests? | Automated tool | Commissioned specialists | External researchers, often via a platform |
| What is found? | Known vulnerabilities, outdated versions, misconfigurations | Exploitable vulnerabilities, logic flaws, chains | What researchers find and report, often the unusual |
| Depth | Broad but superficial | Deep within the agreed scope | Varies, depends on participants |
| False alarms | Frequent | Rare, findings are evidenced | Many reports that have to be assessed |
| Plannability | High, repeatable at will | High, fixed period and scope | Low, reports arrive continuously |
| Prerequisites in the company | Someone to assess results | Agreed scope, test environment or maintenance window | Mature process for assessment, remediation and reward |
| Typical use | Ongoing, e.g. monthly or after changes | Before launch, after major changes, regularly | Complementary for mature, exposed applications |
None of the methods replaces the others. The scan shows a lot in a short time, the pentest shows what really counts, and bug bounty finds what nobody thought of.
With scanners, a second look at the type of scan is also worthwhile. An outside scan sees only what every visitor sees and often has to guess versions from the server’s responses. An authenticated scan, in which the tool reads the installed packages on the server using its own credentials, delivers considerably more reliable results, especially for the actual package level of PHP. It does presuppose, however, that you set up and well protect a restricted account for it. For the dependencies of your own PHP applications there are in turn dedicated checking tools, such as composer audit for Composer packages.
The sensible order for SMEs
- Inventory and scan: First know which websites and servers exist, then scan regularly. Do not adopt findings unchecked, but assess them.
- Fix the basics: Remove outdated software, missing updates and obvious misconfigurations. Commissioning a pentest on unpatched systems wastes budget.
- Targeted penetration test: For the most important applications, such as shop, customer portal or interfaces, with a clearly agreed scope and a retest.
- Code review for in-house developments: Anyone running their own PHP applications supplements the test with a review of the source code. How this works is described in Security code review for PHP projects.
- Reporting point for vulnerabilities: A policy on how outsiders can report vulnerabilities, and a security.txt file under RFC 9116 at
/.well-known/security.txtwith a contact and expiry date. - Bug bounty, once the basics are in place: Only when reports are reliably assessed and vulnerabilities fixed promptly.
To turn this into a lasting process, take a look at Vulnerability management for mid-sized businesses.
What none of the methods achieves
No testing method proves that a system is secure. It only shows what was found at one point in time with certain methods. A clean pentest report from spring says nothing about a vulnerability like PHP2Shell that becomes known in October.
Nor do a scan or pentest answer the question of whether a server is already compromised. That requires a search for traces on the system itself: processes, files, cron jobs, accounts, connections. That is a task of its own with its own methods.
Have only systems tested that you are authorised for, and for hosted systems clarify the provider’s rules beforehand. A penetration test needs a written commission with scope, period and contacts.
When you do not need an external test
For a simple company website without forms handling sensitive data, at a host that installs updates reliably, a regular scan with a critical assessment of the results is often enough. A penetration test would usually be disproportionate here. More important are up-to-date software, strong credentials and backups.
An external test is worthwhile for shops, portals, in-house developments, interfaces to internal systems and wherever customers or standards demand independent evidence.
How does PHP SOS help?
- We set up regular scans and assess the results, including false alarms caused by backports.
- We help to fix the basics before money flows into a pentest.
- Together with specialised partners, we carry out security audits and penetration tests with an agreed scope and retest.
- We support the setup of a reporting point for vulnerabilities and a security.txt.
Conclusion
Scan, pentest and bug bounty are stages, not competitors. Anyone who starts with regular scans, fixes the basics and then has targeted tests carried out gets the most security for their budget. Bug bounty is the freestyle for organisations whose compulsory exercises are already in place.
Frequently asked questions
What is the difference between a pentest and a vulnerability scan?
A vulnerability scan checks automatically whether known vulnerabilities or misconfigurations are detectable. A penetration test goes further: specialists try, within an agreed scope, to actually exploit and chain vulnerabilities, and assess the consequences.
Can a scanner find out whether my server is patched against PHP2Shell?
Only to a limited extent. Many scanners read the displayed PHP version. If the distribution installs a backport, the number often stays the same, and the scanner reports a vulnerability that was closed long ago. Conversely, a new number may be displayed while old processes are still running.
Is a bug bounty programme something for small companies?
Usually not yet. Bug bounty presupposes that reports are assessed quickly and vulnerabilities fixed promptly. A good start is a simple policy for reporting vulnerabilities with a security.txt file.
May I scan my own website with a scanner?
Your own systems, as a rule, yes, if you are authorised to do so. If the website is with a host, clarify beforehand whether scans are permitted or must be announced. Test other people’s systems only with the operator’s written permission.
How often should a pentest take place?
There is no fixed rule. A test before the launch of a new application, after major changes and at regular intervals determined by your risk and, where applicable, requirements from customers or standards is usual.