Experienced Software Engineers: Coding Tasks for AI Systems

Accountabilities: As an Experienced Software Engineer contributing to AI system evaluation, you will design and validate complex coding tasks that simulate real-world software engineering challenges. You will create reliable testing environments and reference solutions that allow AI systems to be evaluated accurately and consistently. Design realistic, self-contained software engineering problems for AI coding system evaluation. Create challenging development scenarios that require advanced problem-solving and engineering skills. Configure and manage development environments, including working with Docker containers. Navigate, analyze, and debug unfamiliar codebases independently. Write robust, deterministic automated tests to assess whether solutions meet task requirements. Develop verified reference implementations to ensure accurate grading and evaluation. Validate that coding tasks are clear, reproducible, solvable, and appropriately challenging. Use GitHub workflows and command-line tools to manage development tasks effectively. Apply software engineering best practices when creating technical evaluation materials. Work independently while maintaining high standards of quality and reliability. Requirements: The ideal candidate is an experienced software engineer with strong technical fundamentals and the ability to independently solve complex engineering problems. You should be comfortable working across different codebases, building automated tests, and using development tools to create reliable technical solutions. Professional experience working as a software engineer. Strong proficiency with command-line tools and terminal-based development workflows. Experience using GitHub and modern software development practices. Ability to independently understand, debug, and modify unfamiliar codebases. Strong programming skills and experience building production-quality software. Experience writing clean, reliable, and maintainable automated tests. Familiarity with Docker, system configuration, and development environment setup. Strong analytical and problem-solving abilities. Ability to work independently and manage project deliverables effectively. Willingness to complete identity verification and a technical skills assessment. Experience with complex debugging, automation, or AI-related development projects is a plus. Benefits: Fully remote contract opportunity. Flexible work arrangement with the ability to work independently. Competitive compensation of $82 per completed task. Opportunity to contribute directly to the improvement of advanced AI systems. Work on challenging software engineering problems in an innovative environment. Flexible project-based engagement suitable for experienced engineers. Opportunity to apply technical expertise beyond traditional software development roles. Gain experience contributing to the evolution of AI coding technologies.