For a high-level overview, take the Feature Tour on our website.
For more detail, see the The Coveralls UI in our documentation.
For a quick look at a live project, visit a public repo, like this one.
(Click around and get a feel for the UI.)
The best way, though, is to create an account for free and just get started!
To send coverage reports to Github as PR comments.
Unfortunately, Github doesn’t have a read-only OAuth scope for private repos, so this is the only way we can see which private repos you have access to. More here.
This is a big question with a lot of nuances, but we’ll keep it simple and start from the beginning:
On the back-end:
When you send a coverage report from a CI build to coveralls.io, the Coveralls API reads and stores your coverage stats. Important: We do not store—or even read—any source code, just the stats from your coverage report(s).
Depending on the number of jobs in your build—more-than-one if your build is parallel—Coveralls stores each coverage report you send, then, when finished, merges the coverage stats, job-by-job, file-by-file, and calculates an overall—or aggregate—coverage percentage for your full build.
On the front end:
The Coveralls UI displays this coverage report as a snapshot in time for one build of your project, from its aggregate coverage, down to the coverage for each job, each file, all the way down to the coverage stats for each line of each file, called, if relevant, a hit or miss.
Over time:
This process is repeated for each coverage report you send. So for each new CI build, Coveralls generates a new Coveralls build, displaying each new coverage report in series next to all the others, showing how your project’s coverage changes over time.
In detail:
Moreover, in addition to showing coverage changes between sequential builds, Coveralls compares pull request builds to their base builds in a deeper manner:
For PR builds, Coveralls highlights two key diffs that matter to developers:
Actionable:
PR diffs show developers the exact places they can improve their PRs to keep a project’s aggregate coverage moving in the right direction.
Visual:
To see how all this information appears visually, check out The Coveralls UI.
Not at this time.
Currently, the Coveralls API has two (2) endpoints:
Neither of these endpoints has the coverage data you’ll find in Coveralls build pages, or in Coveralls Notifications.
It’s definitely one of our goals to add a /builds endpoint to the API, but in the meantime, there are two ways to get coverage data out of Coveralls besides the Coveralls UI and Coveralls Notifications:
Crawl the web app - Nearly every page of coveralls.io has a JSON version from which you can obtain coverage data. (Here’s an example)
Webhook Notifications - You can configure your repo to send coverage reports in JSON format to any endpoint you create to receive them.
First, check our status page for any issues or updates.
Next, read about some common issues related to performance.
To find out more about what might be going on behind-the-scenes, see the explanation here.
To understand what you might be able to do about it, see, How can I speed up my build times?, below.
If your build times seem slow, there may or may not be something you can do about it. First…
If you’re on Coveralls Enterprise:
Good news. You have complete control over the server and network resources running your instance of Coveralls.
First, make sure you’re at least meeting our recomended minimum server requirements.
Next, ask your devops team to increase resources on your Coveralls server, or provision a new server, then use your dedicated Slack support channel, or email us at support@coveralls.io, and we’ll walk you through reinstalling your license.
If you’re on Coveralls Cloud:
Coveralls Cloud (coveralls.io) runs on shared resources. So, while all your data is isolated and secure there, the background jobs that construct your builds run on the same infrastructure that serves other projects.
This means that traffic can affect build times and that, during periods of high traffic, there’s not not a lot you can do to improve your own build times.
That said, if your build is parallel—composed of several jobs—you may be able to speed up your build times by merging your coverage reports in CI and sending one report to Coveralls in a single job.
Upon combining reports in CI your reduction in build times should be roughly linear. Which is to say, you may be able to cut build times in half if you had two jobs, by two-thirds if you had three jobs, and by 90% if you had ten.
Mileage may vary—and will—as the main factor impacting build times is traffic.
Email us at support@coveralls.io if you need some guidance on approach.
Coveralls Cloud, is the service at coveralls.io that’s available to all users of Github, Gitlab or Bitbucket. It is free for public repos and available in several pricing plans based on number of private repos.
Coveralls Enterprise is a private, on-premise version of Coveralls that’s installable on your own servers and network. Its license fees are based on number of users.
Free trials are available for both versions of Coveralls.
Both versions of Coveralls offer feature parity—with the exception that Coveralls Enterprise includes an admin panel for configuring third party integrations.
Support is available for both versions of Coveralls at support@coveralls.io.
See the next question for more about support for Coveralls Enterprise.
With Coveralls Enterprise, you have total control over your server and network environments, but both will be your responsibility.
On the other hand, Coveralls will support you throughout your installation until you’re fully up-and-running, and will release upgrades into a channel to which you are subscribed, allowing you to update features when you’re ready.
We will always address bugs or other issues you encounter with our software, and end user support will always be available to you during business hours via a dedicated Slack channel in the Coveralls workspace and via support@coveralls.io.
For Coveralls Cloud, we accept payment by credit card and ACH debit, either monthly or annually (10% discount for annual payment). Your payment method is stored and automated payments are charged at the beginning of each subscription period.
For Coveralls Enterprise, in addition to payment by credit card and ACH debit, we can perform invoice-based billing and accept checks or money orders.
We can do invoice billing for Coveralls Enterprise.
We cannot offer invoice billing for Coveralls Cloud. We may infequently make an exception upon request, but only for annual payment plans.
Email us support@coveralls.io and we’ll share with you the answers to our latest industry-standard security survey, along with a series of documents on our security policies and procedures.
Why don’t you just publish those?
Some of the information is confidential so we need to keep a log of everyone with whom we’ve shared these docs.
We are not SOC2 certified, but we are aligned with the majority of SOC2 controls, which you’ll find documented in our security policies and procedures.
Our independent security consultants have advised that SOC2 is not a requirement for Coveralls based on the types of data we do—and more importantly do not—store.
Customers are sometimes surprised to hear that we don’t store any source code on Coveralls servers, at any time, and, in addition, we store no personal information about your users, including logins. This is because the only way users access coveralls.io is via OAuth through one of the three major repo hosting services: Github, Gitlab or Bitbucket.
Unfortunately, we cannot fill out third-party security surveys. We explain why, here.
That said, it’s easy to learn more about Coveralls security practices.
There is no team, or member, or user management in Coveralls!
Anyone who has access to your repo on GitHub, et. al. will be able to see its coverage information at coveralls.io.
If the other members of your team have been added to your repo at GitHub, et. al. but can’t see it at coveralls.io, have them visit this link to refresh their GitHub scope. More here.
You’ll need to grant commenting access to our commenter bot at https://github.com/coveralls.
For more on this, see PR Comments in Coveralls Notifications.
Yes we do!
Visit Parallel Builds to learn more about configuring your repo for parallel builds.
Yes! You’ll find it at: http://status.coveralls.io.
We also tend to tweet about issues as they occur, so follow us on Twitter.
We do. Lets discuss. Please email us at sales@coveralls.io.
Please use our public issues board.
For problems with coveralls.io, or the Coveralls API, please open a public issue.
For problems with a Coveralls Integration, please open an issue at your integration’s project home.
For help determining where an error originates, see the Troubleshooting section of Common Issues & Troubleshooting.
Deleting all your repositories, or your account, does not cancel your subscription.
If you no longer need a subscription to track coverage data on your private repos, here are the steps to cancel:
If you’ve lost contact with the owner of your subscription, email us at support@coveralls.io and we’ll be happy to transfer the subscription, or cancel it for you.
To qualify as “open source,” a repo must simply be publicly accessible at its repo hosting service (Github, Gitlab or Bitbucket).
We don’t distinguish between the various licensing models of individual projects. Doing that would be counter to our mission to provide a free test coverage service to the open source software community, and—as part of that—to be a platform for learning about, sharing and improving the quality of code.
Yes we do!
If you create a new Coveralls Integration for your language community, we will happily show our appreciation for your effort with a year of free Coveralls Cloud service for your organization.
If you contribute to a Coveralls Integration, or create Coveralls documentation, we will gladly reward your effort with one or more months of free Coveralls Cloud service.
Any problems, questions or comments about this doc? Let us know.