AI Engineer - LLM Applications Specialist

AI Engineer - LLM Applications Specialist

About the Company:

Great Question is the all-in-one customer research platform loved by teams from Intuit to Amazon, Canva and Figma. We're now looking to bring on an AI Engineer to help us enhance our platform with advanced AI capabilities, creating intelligent research assistants that will revolutionize how our customers conduct research.

We're a team of 30+, led by second time founders, who have 3x revenue through 2024, and we're looking to do it again in 2025. This is an opportunity to get in early, work on an exciting AI implementation, have a huge impact on the technology of an early-stage company, and shape the future of AI-assisted customer research.

This is a fully remote role open to people in the US or Canada with a strong preference to one of our hubs of Denver/Boulder Colorado, SF Bay Area California, Raleigh North Carolina; or Toronto Canada.

Why join us?

At Great Question, we're building a culture of collaboration, curiosity, and impact.

By joining our team, you will:

• Work closely with a passionate, mission-driven team

• Have a direct impact on the growth and success of the company

• Be at the forefront of implementing practical AI solutions in a SaaS product

• Shape the direction of our AI strategy and implementation

• Learn from experienced startup founders (10 years experience, 2 previous exits)

• Grow with the company - opportunity to specialize deeply in AI engineering or move into technical leadership

Why not join us?

At Great Question, we believe in transparency—not just about our strengths, but also about where we still have room to grow. Here's what you should know before deciding to apply:

• Evolving AI Strategy: Our AI implementation is still in its early stages, so expect to help define approaches and best practices

• Lean Teams, Big Impact: You'll wear multiple hats in the AI development process, from prompt engineering to implementation and evaluation

• Fast-Paced Growth: We move quickly with high expectations, requiring creativity and comfort with iteration and occasional ambiguity

We're pretty far from perfect, and we don't claim to be. But we'd like to get there. So if you're someone who thrives on solving complex AI problems, enjoys creating intelligent systems from scratch, and loves being part of a dynamic, fast-growing environment, Great Question might be the right place for you.

About the Role:

Role Overview

We are seeking a highly skilled AI Engineer to design and implement an agent-based UI on top of our existing customer research platform. You will be focused on creating an AI research assistant that can help our users create research studies, review documents, and act as an intelligent research companion. The ideal candidate will have expertise in prompt engineering, agent frameworks, and LLM application development with a focus on creating production-ready AI solutions in JavaScript.

Key Responsibilities

AI System Architecture & Development:

• Design and build agent-based flows that integrates seamlessly with our existing platform

• Develop sophisticated prompt templates that effectively interact with our API to create research studies, review documents, and assist in research activities

• Evaluate and recommend the optimal framework for our use case (LangGraph vs. alternatives)

• Implement the chosen agentic framework to create a coherent AI agent experience

• Ensure the AI system can effectively understand user research needs and translate them into actionable research plans

Prompt Engineering & Optimization:

• Create robust, reliable prompts that consistently produce high-quality outputs

• Develop techniques to ensure AI responses remain factual, helpful, and aligned with research best practices

• Optimize prompts for different use cases within the customer research workflow

• Document prompt strategies and engineering decisions for knowledge sharing

Framework Selection & Implementation:

• Provide expert guidance on the selection of appropriate agentic frameworks (whether LangGraph or alternatives)

• Explain trade-offs between different frameworks in terms of performance, maintainability, and integration with our JavaScript stack

• Stay current with the latest advancements in LLM agent frameworks and recommend improvements

• Implement the chosen framework effectively within our JavaScript environment

Evaluation & Quality Assurance:

• Design and implement comprehensive evaluation frameworks to measure AI system performance

• Create automated tests to ensure consistent AI behavior across different scenarios

• Develop monitoring systems to track AI system performance and identify areas for improvement

• Collaborate with the product team to gather user feedback and iterate on AI capabilities

Technical Integration:

• Work with our strong frontend resources and app developers to integrate AI capabilities

• Focus primarily on the prompting strategy and agentic framework rather than frontend implementation

• Ensure AI features are developed in JavaScript to align with our tech stack

• Document technical specifications and implementation details for the broader team

Qualifications

• Bachelor's degree in Computer Science, AI, or related field

• 5+ years of experience in software development, with at least 1-2 years focused on LLM applications

• Strong programming skills with experience implementing LLM applications (JS preferred)

• Demonstrated expertise in prompt engineering and LLM application development

• In-depth knowledge of various agent frameworks (LangGraph, LangChain, AutoGPT, BabyAGI, etc.) and ability to evaluate their strengths and weaknesses

• Experience designing, implementing, and evaluating AI systems in production environments

• Excellent problem-solving skills and attention to detail

• Strong communication skills and ability to explain complex AI concepts to non-technical stakeholders

• Experience with evaluation frameworks and testing methodologies for LLM applications

Nice to Have

• Knowledge of customer research methodologies and practices

• Previous work with research or data analysis tools

• Experience building AI assistants or agents for specific domains

• Understanding of UX principles for AI interfaces

Support Structure

You'll be working closely with our engineering team who will provide support in:

• Strong frontend resources who will handle UI implementation

• App developers who will assist with platform integration

• DevOps infrastructure and deployment

• Web/API programming integration

• Business domain knowledge and customer research expertise

Your primary focus will be on the AI-specific components: prompt engineering, agentic framework selection and implementation, and ensuring the AI performs effectively for research tasks.

Benefits

• Competitive salary

• Flexible remote work arrangement

• Opportunity to work on cutting-edge AI applications

• Potential for long-term relationship based on project success

• Work with a growing startup making real impact in the customer research space

Hiring Process

We endeavor to deliver a great candidate experience, including notifications of your status in the hiring process throughout.

The hiring process includes the following steps and typically happens across 2-3 weeks from first interview:

1. Initial phone screens for shortlisted candidates

2. Technical Interview with CTO, delving into your AI experience

3. Systems Design Interview

4. Coding Interview

5. Interview with Co-Founder & CEO

6. Reference checks

Note - Each interview is sequential, so you'll only move to the next round if you pass the previous one. After each interview, we'll aim to notify you if you're progressing within 3 days of your interview taking place.

How to Apply

If this sounds like the opportunity you've been waiting for, we'd love to hear from you! Please submit your resume, making sure it's a concise representation of your previous work experience.

Where applicable in your resume, please include specific examples of AI systems you've built in JavaScript, with concrete metrics on their performance and impact. We're particularly interested in your experience evaluating and implementing different agentic frameworks.

Note, we will only ever contact you over email using a greatquestion.co email address.