Strengthening Third-Party Application Security

The Challenge

Every application must pass a penetration test before entering the marketplace and be retested annually to remain listed. Premium listings generate revenue for the company, but the testing requirements apply to every tier.

When the company first turned to UltraViolet, its internal security team was already stretched thin. Its red team was focused on black-box testing of live production systems, leaving a gap in the pre-production testing needed to approve marketplace listings.

The Operational Requirements

The company needs testing capacity that meets specific staffing and reporting requirements. Anyone who accesses U.S. tax data must be based in the United States, which rules out offshore delivery for those engagements.

Every assessor must also hold recognized security certifications. For money-movement applications that process high dollar volumes, the company reviews assessors’ resumes and interviews them before approving their participation.

The company also needs formal, well-documented external testing reports that meet its standards for rigor and support internal reviews.

UltraViolet has met these requirements year after year. As the company pursues ISO/IEC 27001 certification, UltraViolet's testing also provides evidence to support that certification work.

The Engagement

Keeping this testing external was a deliberate build-versus-buy decision. Contracted capacity was easier to secure than new full-time headcount, and external testing also supports the company’s certification work.

Today, UltraViolet fills that pre-production testing gap as the company’s Virtual Security Team (VST), providing ongoing application security support as an extension of its product security team.

A dedicated program manager coordinates the work of nine dedicated assessors, managing assessments from intake through completion and overseeing developer coordination.

With roughly 60 assessments active at any time, new assessments begin as others conclude, sustaining a steady testing cadence throughout the year.

The company previously handled developer coordination in-house, but found that it took time away from the security team's other work. UltraViolet now handles most of that coordination, from onboarding and collecting SOC 2 documentation and other artifacts to explaining findings and helping developers complete the steps needed to move forward. This guidance is especially valuable for teams new to security testing.

Every assessment starts with a review of the application: how critical it is, what data it handles, how technically complex it is, and what has changed since any previous assessment. UltraViolet and the company use these factors to define the testing scope.

Applications that handle sensitive data, such as Social Security numbers, receive a full, in-depth assessment, while lower-risk applications receive a narrower, focused assessment. 

Several applications in the marketplace are mobile. For these applications, UltraViolet provides hands-on, manual testing to complement the program’s largely automated assessment coverage. 

"You do a great job of finding vulnerabilities in mobile applications specifically," the security operations manager said. "This is one of the only places where we're doing manual testing, and the reporting we get back is very helpful."

The Results

Stronger remediation and developer requirements

With UltraViolet’s input, the company updated its program to require remediation and retesting of critical and high-severity findings. A developer’s confirmation alone is no longer enough. Applications with critical findings cannot enter the marketplace until those findings are resolved or a formal exception is granted.

UltraViolet's assessors also flagged a gap between what their reports covered and the security requirements the company published for developers. The company used that feedback to update its guidance.

Evidence for leadership and audit reviews

Assessment findings feed weekly risk reporting to leadership. The company tracks remediation SLAs by severity, ranging from seven days for critical third-party findings to 90 days for low-severity findings. The team reports adherence each week to a CISO who reports directly to the CEO. UltraViolet’s structured reports support that tracking and provide evidence for audit reviews.

The company's data analysts also pull assessment findings into operational dashboards through UltraViolet's APIs, helping leadership track risk across the marketplace and make informed decisions. With some other testing vendors, the company receives static PDF reports and must manually re-enter findings as tickets before engineers can act on them.

Continuing the Partnership

For the company’s security operations manager, a former NSA red teamer, finding a vulnerability is the beginning of a much longer process. He values UltraViolet’s support in helping the team work through findings and act on them.

"Identifying the hole is step one in a fifty-step process," the security operations manager said. "Partnership is the most important thing from a business perspective."

Alongside its remediation and retesting requirements, the company is working toward a more current picture of application risk between annual assessments. Both teams see this as an opportunity to strengthen the program.

As third-party developers build more AI-driven features into their applications, they are beginning to request assessments of those components. Both teams are also discussing how the relationship could grow beyond penetration testing into broader offensive and defensive security work.

UltraViolet’s understanding of the company’s requirements and its established relationships with developers provide a foundation for this next phase, helping the teams address new assessment needs while keeping ongoing testing moving.

Build Capacity for Your Application Security Program

Give your team support from dedicated practitioners who manage ongoing testing and developer coordination, helping you keep pace as your application portfolio grows.

Let’s talk about your testing needs and how UltraViolet could support your team.