8 Questions to ask your Application Security Testing Provider !

Choosing the right Application Security Testing Service Provider is not always an easy task. By asking the right questions and knowing what answers to look for, you can conduct the thorough evaluation of the various vendors available in the market and make the most intelligent choice for your business.There are numerous options like buying tools, using cloud based testing providers or the traditional consultants. I have discussed making the right choice in another blog. However, if you decide to choose Application Security Testing consultants, here are the 9 most important questions you should definitely ask based on the top metrics:

Quality

1. What’s the background of the individuals who will conduct the test?

The background of the people behind the Application Security Testing is one of the most vital factors. Some companies do have good processes but still the individual plays the most important role. So ask for the background of the people conducting the Application Security Tests.

2. What is the methodology of Application Security Testing?

Though the person is very critical, the methodology of Application Security Testing plays an equally major role. If there is a standard process, it ensures minimal quality irrespective of the state of the mind of the consultant. You don’t want that his breakup with his girlfriend causing a significant reduction in the quality of testing. There should be checks and balances to ensure quality irrespective of the situation. Different organizations can have different methodology but you need to figure out from methodologies whether key elements like false positives and business logic vulnerabilities are covered.

3. How will he conduct business logic vulnerability testing?

Business Logic Vulnerabilities cannot be detected by scanners. You need very good processes and skills for theApplication Security Testing vendor to assess such vulnerabilities. It is important to know how the vendors shall conduct such testing.

4. Which tools shall they use?

A good automated scanner is very important for coverage. Free and open source tools are not as good in coverage compared to the best of the breed commercial tools. Free tools need heavy human augmentation and there are risks of higher false negatives.. A good application security testing tool that can crawl modern applications and handle javascript well is very critical. There are several other ways to benchmark an automated scanner. Check out our article on benchmarking automated scanner.

Credibility

5. What are the contributions of the testers in security research (vulnerability discovery, research papers, tools, conference presentations etc)?

Everybody can run a tool. But everybody is not a hacker. You have to fight against the hackers out there on the internet. So it is important that you get a person who matches up to that standard. You should ask him about his background in original security research. Did he do something which is worth being presented in Defcon, Blackhat or other similar conferences?

6. How many and what type of Application Security Tests did he conduct before?

It is important to know the prior experience of the vendor in the field of application security testing. Did he conduct DAST, SAST, Architecture Review, Threat Modeling? You also need to check his experience in discovering Business Logic Vulnerabilities. This is one of the graveyards where many consultants fail unless they have proper experience.

Flexibility and Scalability

7. Can the vendor test during non-business hours?

Sometimes it might be critical to conduct test during non-business hours (nights/weekends). You need to select a Application Security Testing Vendor who is flexible enough to handle any such requirements that you may have.

8. Can the vendor meet up to your scalability requirements?

The last but not the least; if you have to test all your applications two times as per their respective release cycle or at least on a quarterly basis, will the vendor be able to meet such volume requirements. Do they have the infrastructure and the people to conduct such numbers of application security tests?

Few more suggestions by readers and community members Credits: Carlos Rodriguez, Milan Danrel

Customer references, with the ability to interview them. What kinds of problems were found by the vendor, and which ones weren’t?
Verification of background checks of the individual tester
Financial statements of the organization.
Which tools are being used by the tester?
Integration capabilities to collaborative solutions, GRC solutions, dashboard solutions, QA solutions & ticketing systems.
Does the vendor meet the compliance specific expectations? (eg. PCI DSS 1.2)?

More: Want to be a infosec community contributor? Click here

All Articles