Ever found yourself pushing code at 2 AM, crossing your fingers, and hoping nothing breaks in production? We've all been there. But as AI accelerates our development speed, the "push and pray" approach isn't cutting it anymore. That's where automated testing comes in - and we're going to show you how to set it up in minutes.

This tutorial is perfect for Python developers who:

• Know their way around Python but haven't tackled testing yet
• Are shipping code faster than ever thanks to AI tools
• Want to catch bugs before users do (and before that 2 AM emergency call)
• Would rather spend time building features than manually writing test cases

We'll show you how to set up a complete testing pipeline powered by AI. CodeBeaver will automatically generate, run, and maintain your tests - think of it as having a testing expert on your team who never sleeps.

The best part? It's completely free for open source projects, and you can try it on private repos with our 14-day trial. No credit card required.

Ready to level up your Python development game? Let's dive in!

Getting Started

We're going to set up a complete Python testing workflow that will:

• Automatically run tests whenever you push new code
• Generate missing tests to maintain solid coverage
• Help catch bugs before they reach production
• Work naturally with your existing GitHub workflow

The best part? Once we set this up, it largely runs itself. No more hastily adding # TODO: add tests comments at midnight before a release.

Prerequisites

Before we dive in, make sure you have:

• Python 3.6 or higher installed (run python --version to check)
• pip installed and working (run pip --version to verify)
• A GitHub account (also GitLab or Bitbucket will work)
• Basic familiarity with Python and pip
• A willingness to level up your development game

What's Coming Up

Think of this tutorial as your journey from "what's a test fixture?" to "look at my beautiful test coverage report!" We'll progress through several key stages:

1. First, we'll set up a basic Python project with proper structure - similar to how you'd set up a new virtual environment and requirements.txt, but optimized for testing.

2. Next, we'll introduce pytest - think of it as pip for testing. Just as pip manages your project's dependencies, pytest will manage your test execution.

3. Then, we'll connect CodeBeaver - your new AI testing assistant. Imagine having a senior developer who specializes in testing looking over your shoulder, but without the awkward coffee breath.

4. Finally, we'll create your first pull request and watch as CodeBeaver automatically generates tests for your code.

Understanding the Value of Testing in Python

Before we jump into the technical setup, let's talk about why testing matters specifically for Python projects. Unlike languages with static typing, Python's dynamic nature means we can't rely on the compiler to catch certain types of errors. Consider this seemingly innocent code:

def calculate_user_score(points, multiplier):
    return points * multiplier

# Works fine
result = calculate_user_score(100, 1.5)  # 150.0

# Uh oh...
result = calculate_user_score("100", 1.5)  # TypeError!

Without tests, this type of error might only surface when a user enters string input instead of a number - possibly weeks after deployment. Good unit tests catch these issues before they reach production, saving you from those dreaded 3 AM "production is down" messages.

Think of unit tests like the docstring checking feature in your IDE - they're there to catch problems early and document how your code should behave. Just as you wouldn't skip writing requirements.txt because managing dependencies manually is "faster," you shouldn't skip testing because writing features without tests feels quicker in the moment.

What Makes Python Testing Different?

Python's testing ecosystem has some unique advantages that make it particularly well-suited for automated testing:

• Python's assert statement makes test assertions readable and natural
• The pytest framework uses Python's introspection capabilities to make test discovery automatic
• Python's dynamic nature means we can easily mock and patch objects for testing
• The extensive use of duck typing in Python makes it crucial to test behavior rather than just types

In the next section, we'll start setting up our Python project with these advantages in mind. We'll create a structure that makes testing as natural as importing a module, and then we'll let CodeBeaver handle the heavy lifting of actually writing those tests.

Remember: Just as pip revolutionized Python package management (remember the days of manually downloading and installing packages?), tools like CodeBeaver are revolutionizing how we approach testing. Ready to join the testing revolution? Let's dive into the setup in the next section!

Setting Up Your Python Project for Testing

Remember how switching from pip install commands to using a requirements.txt file made your Python projects more organized and reproducible? We're about to do something similar for testing. Just as a well-structured requirements file makes dependency management smoother, a well-organized testing structure makes writing and maintaining tests much easier.

Creating Your Python Project Structure

Let's start with a project structure that separates our application code from our tests. While you might be used to seeing Python projects with just a src directory and requirements file, a test-friendly structure includes a few additional elements. Here's what we'll create:

my_project/
├── src/
│   ├── my_project/
│   │   ├── __init__.py
│   │   ├── core.py
│   │   └── utils.py
│   └── setup.py
├── tests/
│   ├── __init__.py
│   ├── test_core.py
│   └── test_utils.py
├── requirements.txt
├── requirements-test.txt
├── pytest.ini
└── .gitignore

Let's create this structure step by step. First, make your project directory and navigate into it:

mkdir my_project
cd my_project

Now, let's set up a virtual environment - this is probably familiar territory for you:

python -m venv venv
source venv/bin/activate  # On Windows, use: venv\Scripts\activate

Understanding the Two-Requirements Approach

You might be wondering why we have both requirements.txt and requirements-test.txt. This separation follows a important principle in software development: keeping production dependencies separate from development dependencies. Here's why this matters:

# requirements.txt - Production dependencies
pandas==2.0.0
requests==2.28.2
python-dotenv==1.0.0

# requirements-test.txt - Testing dependencies
pytest==7.3.1
pytest-cov==4.0.0
pytest-mock==3.10.0
-r requirements.txt  # This line inherits production dependencies

Think of it this way: when your code runs in production, it doesn't need testing libraries. Just as you wouldn't include your development IDE in your production environment, you don't need testing tools there either. This separation also makes your deployment pipelines faster and more secure.

Setting Up pytest

Now, let's create a pytest.ini file. This is similar to how setup.cfg configures your Python package behavior - it tells pytest how to run your tests:

[pytest]
testpaths = tests
python_files = test_*.py
addopts =
    --cov=src/my_project
    --cov-report=term-missing
    --cov-fail-under=80
markers =
    slow: marks tests as slow (deselect with '-m "not slow"')
    integration: marks tests as integration tests

Let's break down what each of these settings does:

• testpaths = tests tells pytest where to look for test files
• python_files = test_*.py defines the naming pattern for test files
• addopts configures default command-line options:
  • --cov=src/my_project measures code coverage for your project
  • --cov-report=term-missing shows which lines aren't tested
  • --cov-fail-under=80 ensures at least 80% code coverage
• markers lets you categorize tests (similar to how you might tag functions with decorators)

Adding Some Initial Code to Test

Let's add a simple Python module that we'll use as an example. Create src/my_project/utils.py:

def calculate_risk_score(income: float, credit_score: int, debt_ratio: float) -> float:
    """
    Calculate a risk score for a loan application.

    Args:
        income: Annual income in dollars
        credit_score: Credit score (300-850)
        debt_ratio: Monthly debt payments / monthly income

    Returns:
        risk_score: Score between 0 (highest risk) and 100 (lowest risk)
    """
    if not 300 <= credit_score <= 850:
        raise ValueError("Credit score must be between 300 and 850")

    if debt_ratio < 0 or debt_ratio > 1:
        raise ValueError("Debt ratio must be between 0 and 1")

    if income <= 0:
        raise ValueError("Income must be positive")

    # Base score from credit score (40% weight)
    credit_score_component = ((credit_score - 300) / 550) * 40

    # Income component (30% weight)
    income_component = min(income / 100000, 1) * 30

    # Debt ratio component (30% weight)
    debt_component = (1 - debt_ratio) * 30

    return credit_score_component + income_component + debt_component

This function might look simple, but it has several aspects that need testing:

• Input validation for each parameter
• Edge cases (minimum and maximum values)
• The business logic of the scoring algorithm
• Error handling

Without proper testing, we might miss edge cases like what happens when someone enters negative income or a credit score of 900. This is exactly the kind of code where CodeBeaver shines - it can identify these edge cases automatically.

Running pytest

Now that we have our project structure set up, let's run pytest to see where we stand with our testing. This step is important because it establishes our baseline and helps us understand what CodeBeaver will be working with.

First, let's install our testing dependencies. Create a requirements-test.txt file with these contents:

pytest==7.3.1
pytest-cov==4.0.0
pytest-mock==3.10.0
-r requirements.txt

Then install these dependencies:

pip install -r requirements-test.txt

Now, let's run pytest for the first time:

python -m pytest

You'll see output that looks something like this:

============================= test session starts ==============================
platform linux -- Python 3.9.7, pytest-7.3.1, pluggy-1.0.0
rootdir: /path/to/my_project, configfile: pytest.ini
plugins: cov-4.0.0, mock-3.10.0
collected 0 items

============================ no tests ran in 0.12s ============================

Don't worry about the "no tests ran" message – this is exactly what we expect! Remember, we haven't written any tests yet. This is where CodeBeaver will come in handy.

Let's also try running pytest with coverage reporting enabled:

python -m pytest --cov=src/my_project

You'll see additional output showing our current coverage:

============================= test session starts ==============================
platform linux -- Python 3.9.7, pytest-7.3.1, pluggy-1.0.0
rootdir: /path/to/my_project, configfile: pytest.ini
plugins: cov-4.0.0, mock-3.10.0
collected 0 items

---------- coverage: platform linux, python 3.9.7-final-0 -----------
Name                                      Stmts   Miss  Cover
-------------------------------------------------------------
src/my_project/utils.py                      20     20     0%
-------------------------------------------------------------
TOTAL                                        20     20     0%

============================ no tests ran in 0.12s ============================

This output tells us several important things:

• Our project structure is correct (pytest found our code)
• We have 0% test coverage (all 20 statements in utils.py are untested)
• The testing environment is properly configured

This is exactly the situation where CodeBeaver shines. Rather than writing all these tests manually, we'll let CodeBeaver analyze our code and generate comprehensive tests that cover all these untested statements. In the next section, we'll push our code to GitHub and set up CodeBeaver to start generating these tests automatically.

The structure we've created gives CodeBeaver everything it needs to understand our code and generate meaningful tests:

• A clear project structure that follows Python best practices
• Well-documented functions with type hints and docstrings
• Properly configured testing tools (pytest and coverage reporting)
• A baseline coverage report showing what needs to be tested

With this foundation in place, we're ready to move on to setting up our GitHub repository and connecting CodeBeaver. This is where the magic begins – transforming our untested code into a well-tested, production-ready project.

Pushing a Pull Request on Github

Before we move on to installing CodeBeaver, let's create a .gitignore file to keep our repository clean. You can copy this one from GitHub. Add it to the root of your proejct.

(Optional) Create a New Repository on GitHub

This section applies only if you don't have a GitHub repository yet for this project. You can skip to the next chapter if you already have a repository.

1. Log into your GitHub account and click the "+" icon in the top right, then select "New repository"
2. Name your repository "python-risk-calculator" (or another name that describes your project)
3. Add a description like "A Python library for calculating financial risk scores"
4. Choose "Public" if you want to use CodeBeaver's free tier for open source projects, or "Private" if you're using the trial/paid version
5. Don't initialize with a README since we already have our local project
6. Click "Create repository"

GitHub will show you commands to push your existing repository. Don't copy them just yet - we'll use our own sequence of commands.

Initialize Git and Push Your Code

First, initialize Git in your project directory and create your initial commit:

# Make sure you're in your project root directory
cd my_project

# Initialize git
git init

# Add all files
git add .

# Create initial commit
git commit -m "feat: initial project setup with risk calculator"

Now, connect your local repository to GitHub and push your code:

# Add the remote repository (replace with your repository URL)
git remote add origin https://github.com/yourusername/python-risk-calculator.git

# Push your code to main branch
git push -u origin main

Create a Feature Branch

Following Git best practices, let's create a feature branch for our risk calculator implementation:

# Create and checkout a new branch
git checkout -b feat/risk-calculator-implementation

Now that we are in a feature branch, let's update our utils.py to include a new helper function:

def validate_loan_parameters(loan_amount: float, loan_term_years: int) -> None:
    """
    Validate basic loan parameters.

    Args:
        loan_amount: The requested loan amount in dollars
        loan_term_years: The loan term in years

    Raises:
        ValueError: If parameters are invalid
    """
    if loan_amount <= 0:
        raise ValueError("Loan amount must be positive")
    if not 1 <= loan_term_years <= 30:
        raise ValueError("Loan term must be between 1 and 30 years")

These changes add some functionality to our risk calculator. Let's commit the changes and push them to GitHub!

# Commit the changes
git add .
git commit -m "feat: add risk calculator implementation"

# Push the branch to GitHub
git push -u origin feat/risk-calculator-implementation

Open Your Pull Request

Now that we have a feature branch pushed to GitHub, we can open a Pull Request. A Pull Request is, in brief, a way to propose changes to a repository.
To open one:

1. Visit your repository on GitHub
2. You should see a banner saying "feat/risk-calculator-implementation had recent pushes" with a "Compare & pull request" button
3. Click the "Compare & pull request" button
4. Fill in your pull request description.
5. Click "Create pull request"

Now your code is ready for CodeBeaver to analyze and generate tests! The pull request we've created follows several best practices:

• It's focused on a single feature
• It has a clear description of what's being implemented
• It mentions testing expectations
• It's based on well-documented code (thanks to our detailed docstrings)

These practices help CodeBeaver generate more relevant and comprehensive tests for your code. In the next section, we'll see how to install CodeBeaver and watch it work its magic on this pull request.

Installing CodeBeaver

Now that we have our Python project structured and ready, let's integrate CodeBeaver into our workflow. This integration will transform your repository from having no test coverage to maintaining comprehensive test suites automatically. The process is straightforward and takes just a few minutes.

Step 1: Authentication and Authorization

First, navigate to codebeaver.ai and select "Sign up with GitHub". This initiates a secure OAuth flow that will allow CodeBeaver to interact with your repositories. If you're using GitLab or Bitbucket, you'll find similar options for those platforms.

After authenticating, you'll be prompted to authorize CodeBeaver's access to your repositories. You'll see an installation screen that allows you to choose between personal and organizational repositories, select specific repositories or grant access to all.

Click "Install CodeBeaver" to proceed. Don't worry about getting the permissions exactly right - you can always modify these settings later as your needs change.

Step 2: Repository Selection

Once authorized, you'll be presented with a dashboard showing your available repositories. This is where you'll select the repository we just created and enable CodeBeaver for it.

The repository selection interface presents a clear list of your repositories, with options to search and filter if you manage many projects. Select your Python risk calculator repository to proceed.

Step 3: Automatic Configuration

CodeBeaver will now analyze your repository structure to determine:

• The programming language(s) in use
• Testing frameworks present (pytest in our case)
• Project structure and dependencies
• Existing test configurations

Based on this analysis, CodeBeaver will attempt to auto-configure itself. For a standard Python project like ours that uses pytest, this process should complete successfully.

If auto-configuration succeeds, you'll see options for how you'd like to proceed with CodeBeaver. Select the Pull Request you just opened before. You are done! CodeBeaver will start working on your Pull Request.

What If Auto-Configuration Fails?

If you are using your own project, it may happen that CodeBeaver will not be able to auto-configure itself. Don't worry! This usually happens when:

• Your project uses a non-standard structure
• You have multiple testing frameworks
• You need custom test commands

In these cases, you can:

1. Check the troubleshooting guide in the CodeBeaver documentation
2. Add a codebeaver.yml configuration file to your repository
3. Contact CodeBeaver support for assistance

With CodeBeaver installed and configured, you're ready to experience automated test generation in action. In the next section, we'll create our first pull request and watch as CodeBeaver automatically generates and maintains your tests.

Your First Test-Driven Pull Request

In around 5 minutes, CodeBeaver will add a comment to your Pull Request. It will look something like this:

If you click on the Pull Request link in the comment you will go to the Pull Request opened by CodeBeaver. You will notice that it’s a Pull Request opened on top of your original Pull Request:

To review the files, you can check out the diff in GitHub (same as any other Pull Request). Click on “Files Changed”

You can now review the code that CodeBeaver wrote for you.

Once you are ready, you can merge the Pull Request opened by CodeBeaver by clicking on the “Merge pull request”

Working with CodeBeaver: Best Practices

Just as following PEP 8 makes your Python code more maintainable, following certain practices with CodeBeaver will help you get the most out of automated testing. Let's explore some key strategies that will help CodeBeaver generate more accurate and comprehensive tests for your Python code.

Documentation is Your Testing Superpower

Remember how helpful a well-written docstring is when you're trying to understand someone else's code? For CodeBeaver, good documentation is even more crucial. Here's why:

# Less helpful for CodeBeaver:
def calculate_tax(amount, rate):
    return amount * rate

# Much better for CodeBeaver:
def calculate_tax(amount: float, rate: float) -> float:
    """
    Calculate tax amount based on a given rate.

    Args:
        amount: Base amount in dollars (must be positive)
        rate: Tax rate as decimal (e.g., 0.07 for 7%)

    Returns:
        float: Calculated tax amount

    Examples:
        >>> calculate_tax(100.0, 0.07)
        7.0

    Raises:
        ValueError: If amount is negative or rate is not between 0 and 1
    """
    if amount < 0:
        raise ValueError("Amount must be positive")
    if not 0 <= rate <= 1:
        raise ValueError("Rate must be between 0 and 1")
    return amount * rate

The second version gives CodeBeaver crucial information about:

• Expected input types and ranges
• Example usage and expected outputs
• Error conditions that should be tested
• Business rules (like valid ranges for tax rates)

Leveraging Real-World Data in Tests

When you have access to real-world data that represents actual use cases, you can help CodeBeaver generate more realistic tests. There are two effective ways to do this:

1. Include examples in your docstrings:

def validate_user_input(age: int, email: str) -> bool:
    """
    Validate user registration data.

    Args:
        age: User's age in years
        email: User's email address

    Example Data:
        Valid cases:
        - age=25, email="user@example.com"
        - age=18, email="student.2024@university.edu"

        Invalid cases:
        - age=12, email="tooyoung@example.com"  # Age below minimum
        - age=25, email="not-an-email"  # Invalid email format
    """

1. Create a separate test data file:

# tests/test_data/sample_users.py
SAMPLE_USERS = [
    {
        "age": 25,
        "email": "john.doe@example.com",
        "expected_valid": True
    },
    {
        "age": 15,
        "email": "invalid@test",
        "expected_valid": False
    }
]

Managing the Testing Workflow

Think of CodeBeaver as a meticulous code reviewer who specializes in testing. Here's how to make the most of this collaboration:

1. Review Generated Tests First Before merging your feature PR, always review and merge CodeBeaver's test PR first. This ensures that:

• Your code is thoroughly tested before reaching production
• You can catch any misunderstandings about the intended behavior
• You might discover edge cases you hadn't considered

1. Iterative Testing If CodeBeaver's initial tests reveal gaps or misunderstandings:
   • Update your docstrings to clarify the expected behavior
   • Add more examples or edge cases to the documentation
   • Push these changes and let CodeBeaver generate updated tests

Special Considerations for Large Codebases

When installing CodeBeaver in an existing large Python project, consider this gradual approach:

1. Start with New Code

• Initially configure CodeBeaver to focus on new PRs and modified files
• This prevents overwhelming reviews with massive test PRs

1. Incremental Coverage

• Gradually expand coverage by adding tests for existing code
• Focus on critical paths and high-risk areas first
• Use CodeBeaver's file-specific testing feature for targeted improvements

1. Documentation Updates
   • Systematically improve docstrings in existing code
   • Prioritize modules that will need tests soon
   • Include real-world examples from your production system

Remember: Just as you wouldn't push to production without reviewing your code, always review CodeBeaver's generated tests. They're not just a safety net – they're documentation of your code's intended behavior and edge cases.

By following these practices, you'll create a virtuous cycle: better documentation leads to better tests, which lead to more reliable code, which makes future development faster and safer. It's like compound interest for your codebase's quality!

Looking Forward

As your test suite grows, consider setting up test coverage goals in your pytest.ini file:

[pytest]
addopts =
    --cov=src/my_project
    --cov-report=term-missing
    --cov-fail-under=90  # Gradually increase this as coverage improves
markers =
    api: marks tests that require API access
    integration: marks integration tests

This gives your team clear targets while letting CodeBeaver help you reach them systematically and efficiently.

Remember: The goal isn't just to have tests – it's to have meaningful tests that catch real issues and document your code's behavior. With these practices and CodeBeaver's help, you're well on your way to achieving both!

Conclusion

Congratulations! You've just transformed your Python project from "we'll add tests someday" to "fully tested and ready for production." Just as virtual environments revolutionized Python development by making dependency management painless, CodeBeaver is changing how we approach testing by making it automatic and maintainable.

Throughout this tutorial, we've seen how combining good Python practices with AI-powered testing can create a robust development workflow. We started with a basic Python project structure, added proper documentation, and let CodeBeaver handle the heavy lifting of test creation and maintenance. The result? A testing pipeline that catches bugs before they reach production, without requiring you to become a testing expert overnight.

The real power here isn't just in having tests – it's in having a sustainable, scalable way to maintain test coverage as your codebase grows. Just as Git tracks your code changes, CodeBeaver now tracks your testing needs, automatically updating and expanding your test suite with each pull request.

Want to explore more automated testing approaches? Check out our React testing tutorial, which shows how to set up a similar pipeline for frontend development. Whether you're working with Python, JavaScript, or both, CodeBeaver has you covered.

Remember: Good tests aren't just a safety net – they're documentation, quality control, and peace of mind all rolled into one. With CodeBeaver watching your back, you can focus on what you do best: building great software.

Happy coding, and may all your tests be green! 🚀