We are sharing a specialised remote consulting opportunity for experienced Application Security Professionals with strong expertise in secure software development, vulnerability assessment, LLM security, AI-powered static application security testing, and agent-based security architecture to contribute their technical knowledge to advanced AI training and evaluation initiatives.
Selected professionals will design realistic application security evaluation scenarios, develop controlled testing fixtures, assess secure and vulnerable code implementations, and create structured evaluation materials for advanced AI systems. The work combines practical application security engineering with deterministic testing, synthetic vulnerability assessment, technical documentation, and security-focused AI benchmarking. Prior experience with AI training projects is not required.
Key Responsibilities
Application Security Engineering & Vulnerability Assessment
- Design and evaluate realistic application security scenarios involving modern software architectures and secure development practices
- Analyse application codebases for security weaknesses, implementation errors, and potential vulnerabilities
- Assess secure and vulnerable code implementations using established application security principles
- Apply advanced security knowledge to evaluate vulnerability identification, remediation approaches, and secure coding practices
- Develop technically rigorous application security challenges reflecting real-world engineering and security assessment requirements
LLM Security & Agent-Based Architecture
- Evaluate application security risks associated with large language models and AI-powered software systems
- Design controlled security evaluation scenarios addressing relevant OWASP LLM risks, including Sensitive Information Disclosure and Excessive Agency
- Assess security considerations involving agent permissions, tool access, data handling, and system boundaries
- Review security architecture concepts for traditional applications and AI agent-based systems
- Contribute expert insights to the evaluation of secure AI application design and agent security practices
Synthetic Security Fixtures & Deterministic Testing
- Design and implement small, purpose-built application fixtures containing controlled security vulnerabilities or secure reference implementations
- Develop deterministic test cases to support consistent and reproducible application security evaluations
- Create structured security-context manifests describing relevant application conditions and evaluation requirements
- Prepare detailed reference solutions and evaluation criteria to distinguish vulnerable implementations from secure alternatives
- Assess whether synthetic security tasks accurately represent meaningful and realistic application security challenges
AI-SAST Evaluation & Security Benchmarking
- Evaluate AI-assisted static application security analysis methodologies and their effectiveness across controlled testing scenarios
- Review AI-generated vulnerability assessments, secure coding recommendations, and application security findings
- Identify inaccuracies in security classifications, technical reasoning, remediation suggestions, and evaluation outputs
- Refine security benchmarking tasks to improve their technical relevance, consistency, and assessment quality
- Provide structured expert feedback supporting improvements in AI-based application security analysis and vulnerability evaluation
Security Documentation & Collaborative Quality Assurance
- Produce clear technical documentation covering security fixtures, testing methodologies, assessment criteria, and evaluation findings
- Review security evaluation materials for accuracy, completeness, reproducibility, and technical clarity
- Communicate detailed application security insights through structured written feedback and professional discussions
- Collaborate with technical contributors to review security assessments and improve evaluation workflows
- Support continuous improvement of application security testing methodologies, quality standards, and AI evaluation processes
Ideal Profile
- Extensive professional experience in application security, secure software development, cybersecurity engineering, or CyberSWE-related roles
- Approximately 5+ years of relevant professional experience is preferred
- Strong understanding of application security principles, secure coding practices, vulnerability assessment, and remediation methodologies
- Familiarity with OSWE-level application security concepts, advanced web application testing, and practical vulnerability analysis
- Strong knowledge of LLM security risks and emerging security considerations involving AI-powered applications
- Experience with AI-assisted static application security testing (AI-SAST) or related security evaluation methodologies
- Demonstrated understanding of OWASP application security guidance and relevant OWASP LLM security categories
- Experience designing, reviewing, or evaluating security architecture for traditional software applications or AI agent-based systems
- Practical expertise in developing, securing, testing, and reviewing application codebases
- Ability to identify complex application vulnerabilities and evaluate technically appropriate remediation approaches
- Familiarity with security considerations involving agent permissions, tool integration, data exposure, and application trust boundaries
- Experience creating controlled security testing scenarios, synthetic vulnerability fixtures, or reproducible evaluation environments is highly desirable
- Strong understanding of deterministic testing, structured assessment criteria, and technical validation methodologies
- Ability to distinguish secure implementations from vulnerable code using clear, technically justified evaluation criteria
- Strong analytical reasoning, technical problem-solving, and security assessment capabilities
- Excellent written and verbal communication skills, including the ability to explain sophisticated security concepts clearly
- Experience producing detailed technical documentation, security reports, or structured engineering feedback
- Ability to collaborate effectively with multidisciplinary technical teams in a remote environment
- Strong attention to detail and commitment to accurate, reproducible, and actionable security assessments
- Prior experience contributing to AI training or model evaluation projects is not required
Engagement Details
- Freelance / contract engagement
- Fully remote
- Compensation: $40–$80/hour
- 10 available openings
- Specialised consulting role spanning application security, LLM security, AI-SAST evaluation, secure software development, and agent-based security architecture
- Work will involve synthetic security fixture development, controlled vulnerability assessment, deterministic testing, and structured AI evaluation
- Technical Focus: OWASP LLM risks, application vulnerability analysis, secure coding, and AI-assisted security assessment
- Assignments will require clear technical documentation, reproducible testing methodologies, and accurate security evaluation criteria
- Professionals will contribute to the assessment and improvement of security-focused AI evaluation workflows
- Collaboration may involve application security engineers, software developers, AI specialists, and multidisciplinary technical contributors
- Previous experience with AI training projects is not required
- Work must be completed without using confidential or proprietary information belonging to any employer, client, institution, or other third party
About the Platform
This opportunity is available through 24-MAG LLC. We connect experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams.
By submitting this application, you acknowledge that your information may be processed by 24-MAG LLC for recruitment and opportunity matching in accordance with our Privacy Policy: https://www.24-mag.com/privacy-policy
