Get Started

Integrating with Coveralls

Coveralls Builds

The Web App

The API

Get Help

Frequently Asked Questions

Questions

ALL QUESTIONS

How can I find out more about Coveralls features?

For a high-level overview, take the Feature Tour on our website.

For more detail, see the The Coveralls UI in our documentation.

For a quick look at a live project, visit a public repo, like this one.
(Click around and get a feel for the UI.)

The best way, though, is to create an account for free and just get started!

Why does the Coveralls bot need write access to my repo?

To send coverage reports to Github as PR comments.

Unfortunately, Github doesn’t have a read-only OAuth scope for private repos, so this is the only way we can see which private repos you have access to. More here.

How does coverage work? How do I understand these changes in coverage?

This is a big question with a lot of nuances, but we’ll keep it simple and start from the beginning:

On the back-end:

When you send a coverage report from a CI build to coveralls.io, the Coveralls API reads and stores your coverage stats. Important: We do not store—or even read—any source code, just the stats from your coverage report(s).

Depending on the number of jobs in your build—more-than-one if your build is parallel—Coveralls stores each coverage report you send, then, when finished, merges the coverage stats, job-by-job, file-by-file, and calculates an overall—or aggregate—coverage percentage for your full build.

On the front end:

The Coveralls UI displays this coverage report as a snapshot in time for one build of your project, from its aggregate coverage, down to the coverage for each job, each file, all the way down to the coverage stats for each line of each file, called, if relevant, a hit or miss.

Over time:

This process is repeated for each coverage report you send. So for each new CI build, Coveralls generates a new Coveralls build, displaying each new coverage report in series next to all the others, showing how your project’s coverage changes over time.

In detail:

Moreover, in addition to showing coverage changes between sequential builds, Coveralls compares pull request builds to their base builds in a deeper manner:

For PR builds, Coveralls highlights two key diffs that matter to developers:

  • NEW UNCOVERED LINES - New lines of code that aren’t covered by tests
  • UNCOVERED EXISTING LINES - Lines that were covered before but are uncovered now

Actionable:

PR diffs show developers the exact places they can improve their PRs to keep a project’s aggregate coverage moving in the right direction.

Visual:

To see how all this information appears visually, check out The Coveralls UI.

Can I get coverage data through the API?

Not at this time.

Currently, the Coveralls API has two (2) endpoints:

  • /repos - For CRUD of coveralls repos
  • /jobs - For POST-ing coverage reports for a given repo

Neither of these endpoints has the coverage data you’ll find in Coveralls build pages, or in Coveralls Notifications.

It’s definitely one of our goals to add a /builds endpoint to the API, but in the meantime, there are two ways to get coverage data out of Coveralls besides the Coveralls UI and Coveralls Notifications:

  • Crawl the web app - Nearly every page of coveralls.io has a JSON version from which you can obtain coverage data. (Here’s an example)

  • Webhook Notifications - You can configure your repo to send coverage reports in JSON format to any endpoint you create to receive them.

Build times seem slow. What’s going on?

First, check our status page for any issues or updates.

Next, read about some common issues related to performance.

To find out more about what might be going on behind-the-scenes, see the explanation here.

To understand what you might be able to do about it, see, How can I speed up my build times?, below.

How can I speed up my build times?

If your build times seem slow, there may or may not be something you can do about it. First…

If you’re on Coveralls Enterprise:

Good news. You have complete control over the server and network resources running your instance of Coveralls.

First, make sure you’re at least meeting our recomended minimum server requirements.

Next, ask your devops team to increase resources on your Coveralls server, or provision a new server, then use your dedicated Slack support channel, or email us at support@coveralls.io, and we’ll walk you through reinstalling your license.

If you’re on Coveralls Cloud:

Coveralls Cloud (coveralls.io) runs on shared resources. So, while all your data is isolated and secure there, the background jobs that construct your builds run on the same infrastructure that serves other projects.

This means that traffic can affect build times and that, during periods of high traffic, there’s not not a lot you can do to improve your own build times.

That said, if your build is parallel—composed of several jobs—you may be able to speed up your build times by merging your coverage reports in CI and sending one report to Coveralls in a single job.

Upon combining reports in CI your reduction in build times should be roughly linear. Which is to say, you may be able to cut build times in half if you had two jobs, by two-thirds if you had three jobs, and by 90% if you had ten.

Mileage may vary—and will—as the main factor impacting build times is traffic.

Email us at support@coveralls.io if you need some guidance on approach.

What’s the difference between Coveralls Cloud and Coveralls Enterprise?

Coveralls Cloud, is the service at coveralls.io that’s available to all users of Github, Gitlab or Bitbucket. It is free for public repos and available in several pricing plans based on number of private repos.

Coveralls Enterprise is a private, on-premise version of Coveralls that’s installable on your own servers and network. Its license fees are based on number of users.

Free trials are available for both versions of Coveralls.

Both versions of Coveralls offer feature parity—with the exception that Coveralls Enterprise includes an admin panel for configuring third party integrations.

Support is available for both versions of Coveralls at support@coveralls.io.

See the next question for more about support for Coveralls Enterprise.

What can we expect in terms of reliability and support if we go with Coveralls Enterprise?

With Coveralls Enterprise, you have total control over your server and network environments, but both will be your responsibility.

On the other hand, Coveralls will support you throughout your installation until you’re fully up-and-running, and will release upgrades into a channel to which you are subscribed, allowing you to update features when you’re ready.

We will always address bugs or other issues you encounter with our software, and end user support will always be available to you during business hours via a dedicated Slack channel in the Coveralls workspace and via support@coveralls.io.

What forms of payment do you accept?

For Coveralls Cloud, we accept payment by credit card and ACH debit, either monthly or annually (10% discount for annual payment). Your payment method is stored and automated payments are charged at the beginning of each subscription period.

For Coveralls Enterprise, in addition to payment by credit card and ACH debit, we can perform invoice-based billing and accept checks or money orders.

Can we do invoice billing?

We can do invoice billing for Coveralls Enterprise.

We cannot offer invoice billing for Coveralls Cloud. We may infequently make an exception upon request, but only for annual payment plans.

How can I learn more about Coveralls security practices?

Email us support@coveralls.io and we’ll share with you the answers to our latest industry-standard security survey, along with a series of documents on our security policies and procedures.

Why don’t you just publish those?

Some of the information is confidential so we need to keep a log of everyone with whom we’ve shared these docs.

Are you SOC2 certified?

We are not SOC2 certified, but we are aligned with the majority of SOC2 controls, which you’ll find documented in our security policies and procedures.

Our independent security consultants have advised that SOC2 is not a requirement for Coveralls based on the types of data we do—and more importantly do not—store.

Customers are sometimes surprised to hear that we don’t store any source code on Coveralls servers, at any time, and, in addition, we store no personal information about your users, including logins. This is because the only way users access coveralls.io is via OAuth through one of the three major repo hosting services: Github, Gitlab or Bitbucket.

Will you fill out our security questionnaire?

Unfortunately, we cannot fill out third-party security surveys. We explain why, here.

That said, it’s easy to learn more about Coveralls security practices.

How do I add my team members to a repo on Coveralls?

There is no team, or member, or user management in Coveralls!

Anyone who has access to your repo on GitHub, et. al. will be able to see its coverage information at coveralls.io.

If the other members of your team have been added to your repo at GitHub, et. al. but can’t see it at coveralls.io, have them visit this link to refresh their GitHub scope. More here.

How do I enable pull request comments on my private repo?

You’ll need to grant commenting access to our commenter bot at https://github.com/coveralls.

For more on this, see PR Comments in Coveralls Notifications.

Does Coveralls support parallel builds?

Yes we do!

Visit Parallel Builds to learn more about configuring your repo for parallel builds.

Do you have a service status page?

Yes! You’ll find it at: http://status.coveralls.io.

We also tend to tweet about issues as they occur, so follow us on Twitter.

Do you offer non-profit or educational pricing?

We do. Lets discuss. Please email us at sales@coveralls.io.

Where can I request new features?

Please use our public issues board.

How do I report a problem?

For problems with coveralls.io, or the Coveralls API, please open a public issue.

For problems with a Coveralls Integration, please open an issue at your integration’s project home.

For help determining where an error originates, see the Troubleshooting section of Common Issues & Troubleshooting.

I deleted my repos / account. Why am I still being charged for a subscription?

Deleting all your repositories, or your account, does not cancel your subscription.

If you no longer need a subscription to track coverage data on your private repos, here are the steps to cancel:

  • Login to coveralls.io (as the user who created the subscription)
  • Go to SUBSCRIPTIONS
  • Click the SETTINGS button next to your subscription
  • Click Cancel Subscription in the left nav of the Subscription Page.

If you’ve lost contact with the owner of your subscription, email us at support@coveralls.io and we’ll be happy to transfer the subscription, or cancel it for you.

You say, “Always free for open source.” What qualifies as open source at Coveralls?

To qualify as “open source,” a repo must simply be publicly accessible at its repo hosting service (Github, Gitlab or Bitbucket).

We don’t distinguish between the various licensing models of individual projects. Doing that would be counter to our mission to provide a free test coverage service to the open source software community, and—as part of that—to be a platform for learning about, sharing and improving the quality of code.

Do you incentivize contributors?

Yes we do!

If you create a new Coveralls Integration for your language community, we will happily show our appreciation for your effort with a year of free Coveralls Cloud service for your organization.

If you contribute to a Coveralls Integration, or create Coveralls documentation, we will gladly reward your effort with one or more months of free Coveralls Cloud service.

 


Any problems, questions or comments about this doc? Let us know.