TestRail

Vorteile

TestRail allows us to manage our test cases with great Jira integration.

Nachteile

Some of the admin was a bit cumbersome when we first started such that there wasn't a good split between the person that administers the accounts and who pays for it.

Vorteile

After several years in QA and using TestLink and other tools finally I was able to convince management to use TestRail. That was a game changer - easy to use for manual testing, integration with our automation flow and integration with JIRA saved us a lot of work days.

Nachteile

Sometimes I've experienced issue with JIRA integration (we were using Microsoft Active Directory and password to JIRA was changing each month)

Vorteile

Easy to use, simple and straightforward api's to plug into with good documentation. Love this system to use with both manual and automation.

Nachteile

Running test runs are sometimes a pain but you can still manage.

Vorteile

Jira-Integration is very good
API works fine and is well documented

Nachteile

No special support for BDD ( Behaviour Driven Development)

Vorteile

Easy to use, features and that it can be integrated with jira

Nachteile

It would be nice to have a tool to do automated testing

Vorteile

Test rail is very easy to use from creating the test plan to creating test runs.We can easily pull test cases as per our requirements and execute them as a test run.

Nachteile

Test results uploads of videos is comparatively slow.Scree capture and voice recording features should also be there

Vorteile

1. Ability to create test plans and test case for multiple features at once. 2. Ability to configure test cases in combination for different environments and browsers, OS etc. 3. Ability to present detailed reports to stakeholder and management. 4. Integration with Jira eases the need to map to a user story. 5. The test plan hierarchy helps to create test cases easily and it helps reduce redundancy between test cases. 6. Very easy to navigate using the user interface. 6. Ability to lock a test plan where it can no further be modified.
7. Able to run test cases in multiple environment and check the run history. 8. Able to download and email reports to management.

Nachteile

Searching for a past completed story is difficult unless to navigate from Jira. Marking a test case run as passed has too many steps involved. Id love if it has ability to record screen as well as attaching screenshot to a test step/ I'd also love if I can pass/fail each step of a test case and not the test case as a whole.

Vorteile

The best thing about TestRail is their REST API. Thanks to it we were able to easily implement its integration with our test automation framework. In particular we were able to map test exucution statuses to TestRail test case statuses and allow updating them easily for test reruns. Second good thing is reports they allow to generate. That's what our managers wanted.

Nachteile

Sometimes UI is not intuitive. For instance, it took a while to figure out how to change test cases status in bulk.

Vorteile

i have using this tool for my every day test execution before this i was using our own company proprietary tool and then we switched to TestRail as we switched to agile methodology. i have found this quite useful in agile environment. it is scalabe and can be customised accordingly to suite the customer/test requirement.
it is now integrated to our companies other tools like JIRA(bug tracker tool),github,automated test suites, bitbucket etc and many others and is working flawlessly. Reports generation and tracking the various result is quite simple and have real time insights for these with this tool. We can easily link our test cases, requirements, use cases and stores using this.
Really its a great tool with so many functionalities and i love it. it increase the team collaboration in a great way

Vorteile

Versatile allows you to start projects in multiple styles of SDLC agile to waterfall

Customizations are plenty - if you can code you can extend it

Nachteile

the fact that you have to code to extend - the reporting tools out of the box are lacking

Vorteile

We use it TestRails as a host for our Manual regression suite and report our selenium tests results too using the testrails API. Works very nice.

Nachteile

Out company uses TFS and integration with TFS is not very good. The best we can do is link a failed test to a TFS dev task, and it only goes 1 way.

Vorteile

Ease of use is pretty good. Able to recreate Test Plans pretty easily.

Nachteile

Search function should have been a bit more inclusive. Not able to search in multiple projects.
Defects management is something else that Test Rail could do better.

Vorteile

Once you have got used to the terminology and the way TestRail works, it's an extremely easy use and powerful tool for testing. We use it on an almost daily basis and never come across any issues.

Nachteile

Learning the terminology and the way things work together takes a little time. Also, it appears to be developer by a small team and it takes a long time for features to make it to users, even those requested by a large number of people on their website.

Vorteile

It helps our QA team with managing regression testing between multiple remote QA engineers. Adding new test cases is easy, and being able to go back and see who tested each case is extremely valuable.

Nachteile

Getting all of our test cases moved over to TestRail was a huge undertaking, but once done, maintaining has been easy.

Vorteile

I loved it before it was sold. It was solid, easy to use, and made our work flow simple and efficient. That is no longer the case

Nachteile

Since it was bought out, it is now horrific. Test data disappears or is inaccessible. Test runs disappear. EVERY single release is extremely buggy. And knowing that, they still continue to push the release in waves to other customers instead of fixing high priority issues first. Tickets are opened and we get the canned "we're working on it" but rarely are bugs fixed correctly the first time (if at all). We have had months of work disappear with a single release. We've had things break where we were unable to access our tests for days or even weeks at a time. I went from convincing my entire company to join in to use this product, to now only having 3 of us left who will touch it. It's just sad how far it has fallen.

Vorteile

I think report generation is the strong suit of Test rail, the percentage pie chart of passed, failed blocked cases at a glance make daily reporting for scrum stand up meetings ideal.
Integration with Jira is excellent, although some room for improvement.

Nachteile

Trying to do something simple like import test case steps from a .csv is not so easy. You can import a suite but that is not ideal.
Trying to duplicate cases can be tedious, let's say you have a lot of cases, trying to copy paste them in to various projects is a lot of hassle, it is surprising they have not addressed issues like this after so many years.

Vorteile

First of all, TestRail is an extremely smarter way to track and maintain test cases than using Excel. I like how easy it is to create new projects, create custom fields, create test runs, and generate good looking reports to submit to your managers. The GUI takes a little bit to get used to, but once you get a hand of all of the key-shortcuts moving around the application is easy!

Nachteile

My least favorite part about this software is the JIRA plug in. In order to see the linked test cases from TestRail in JIRA, you need to have a TestRail account. In my company only the testing team had TestRail accounts, while the rest of R&D didn't. While it was helpful for me as a tester to see the linked test cases on a JIRA bug, when we started off with TestRail we had assumed everyone in the R&D department would be able to see this information so we could be more transparent in our testing. My company was not willing to pay for 100 TestRail accounts, so this was a bummer. If yours is then TestRail might be right for you!

Vorteile

For me the best thing about TestRail is the visual representation of the test results in the form of graphs, This gives me a quick and easy idea about the stability of the tested software, another thing that is very useful is the format of test suites and test runs.

Nachteile

I think importing testcases from excel can be made a little easier with better mapping and easy uploads.

Vorteile

Easy to set up and understand. Does not require a lot of in-depth config to hit the ground running.

Nachteile

They have not added new features and improved the product in a very long time. It's only incremental updates and fixes.

Vorteile

The ease of use for maintaining, connecting and sharing test cases are unrivaled. The platform is so feature rich every time I think of a feature I want to request they already have it!

Nachteile

Takes a good amount of time to spin up a valuable framework. I think this can be mitigated by some test case best practices or maybe a super-user demo within the platform.

Vorteile

TestRail's simple UI and functionality is straight forward. Easy for people to jump in and use the tool, assuming that the tool fits Dev/QA's team workflow and method of usage. For the administrator, access is easily manageable, and provides multiple categories for restricting access and defining user roles within the organization.

Nachteile

TestRail is no different than other test management systems, beside the front end and UI benefits. TestRail is good for teams who initiate test planning, test case creation, and management specific to older QA management styles. When introducing continuous integration testing into a development/QA' teams methodology, this is where TestRail needs improvement . TestRail does not do a good job logging individual test runs for historic tracking, DB backend cannot handle the data that is pushed to the system, and we often have to delete DB records to help with performance and lookup. Usually without these backend hacks, the system performs very poorly and is slow. TestRail may be good for teams that have not adopted advanced development and QA practices, but if teams want to integrate CI testing with test cases and suites within TestRail this is where we find it to fail.

Vorteile

Great product allowing us to keep history of our testing and specifically regression testing of our software development team

Nachteile

Find some reporting can be difficult, but in general it is a great tool for managing our regression testing test cases, and ensuring that they have all been run.

Vorteile

I'm using this tool everyday at my work, create test cases, tracking their status, and also create different reports to see the status. This tool works fast, has good user friendly design, by tracking results you can get an email notification.

Nachteile

Didn't find any drawbacks its really fast and convenient in using, also help to keep all info in one place.

Vorteile

Its very good for managing test cases modules wise. And provides API also yo integrate with your automation code.

Nachteile

Cost wise its very high. Integration with jira not providing that much of features as compare to Xray.

Vorteile

Its very easy to add the test scenarios in the existing Test Run. We can create multiple Test runs and could create a lot many reports with different filters. As an admin too its very easy for me to give specific access to different members in my team.

Nachteile

There should be some report format where I could show the Stories, linked Test scenarios, linked defects and their status in more readable way. As of now its there but I feel their is a need of more clarity in the reports.

Vorteile

Ease of use. Simple and intuitive flow for the workflows that can be customized based on your work.

Nachteile

Traceability is based on how you configure. Little confusing if you try to explain to users not familiar with the tool.

Vorteile

*customized for any project
* easy to get reports from
* web based
* can import from variety of other tools
* good forums for help and ideas

Nachteile

* can easily make it complex for others depending on how you setup
* a few known issues that can cause data loss

Vorteile

- Web based - doesn't require a localized server installation (unless you want to) and can be accessed by anyone with credentials, no matter where they are located without any effort
- Very easy to use and gives you a very neat category division when it comes to test cases, test runs and milestones

Nachteile

Can't search and replace words in multiple test cases at the same time which makes it harder to re-use existing test cases with only small changes

Vorteile

It is very intuitive. If one used other such tools for test documentation and tracking, it really very use to start with Test Rail.

Nachteile

1) Configuration of the tool is a bit too rigid. I would make it more streamlined and have some admin features for configuration made available for regular users
2) Errors. Rather errors explanations. If possible to expand on what the exact issue is when we get an error.

Vorteile

It's very easy to use, with different kind of configurarions and customizations.

Nachteile

The tests scenarios are not versioned. There's not possibility to see previous changes or have different versions of them.

Vorteile

- easy to use
- you can create a lot of testcases but you still keep the overview
- you can connect it with jira
- after some time the UI is easy to use and you can give the test case to tester

Nachteile

- design is outdated and could be a bit more intuitiv

Vorteile

1- versatility to create Projects which allow base line Test Cases to be used in a variety of different test runs for different products.
2- ease of being able to import test cases from existing methods

Nachteile

The main issue I have is the Projects are listed as a flat-file. We have several different groups which have created Projects and even though they might be testing a given entity from different sides, they have to name their Projects differently. We prefix each project with a Group Name so the Projects can be "grouped" and easily referenced from the main dashboard.
Another less important issue is there isn't really an easy way to make a backup of our entire Dashboard. We can Export each Project but I don't see any way to make backups of anything else outside of offloading each individual entity which really isn't all that hard.

Vorteile

- clean interface
- a few ways to equip/start projects
- a lot of integrations
- available API

Nachteile

- so as much as it is adaptable by doing so it also makes it not the strongest tool since it needs to support many different styles
- reporting engine to do customizations is a learning curve

Vorteile

Easy to see status of projects on dashboard, test runs are quick and easy to set up. DB is backed up, but also the availability to make local backups as well.

Nachteile

Can be a little tricky to get the project set up and configured at first but soon becomes intuitive

Vorteile

It's pretty easy to learn and very quickly onboarding it

Nachteile

No slack integrations very bad no have many integrations with other platforms

Vorteile

User Friendly, Simple UI. Report creation is easy.

Nachteile

Searching ability can be expanded to include more parameters. Sometimes cannot find a test case.

Vorteile

One thing I like the most about the software is the ability to create and share reports on test results.

Nachteile

The thing I like least about the software is the test-case review process is not very streamlined.

Vorteile

The software's very integrated with JIRA

Nachteile

They could improve testing matrixes, reporting like KPIs, Test Coverage, and really need a link between requirements/stories and code coverage.

Vorteile

User friendly, easy to use and to customize, fast and available as a service.

Nachteile

It's getting more expensive, especially when comparing 2018 to 2019 costs (47% increase)

Vorteile

TestRail Case managment software helps a lot with manage of the testing process.

Nachteile

Missing possibility to set default tester per test case.

Vorteile

easily use, friendly UI, open API for integration and fast customer response.

Nachteile

performance issue for large company & heavy users

Vorteile

They provide a lot of resources to guide you on getting started. There are so many ways to set it up to make it work just for you. Infinite ways to organize and plan test runs

Nachteile

Fogbugz integration didn't seem to work very well when we tried.

Vorteile

It plays very well with Atlassian Software Jira.

Nachteile

We have not had any bad experience with the software.

Vorteile

The fact that it's a well-thought-out way to organise test cases and has support for automated tests

Nachteile

Not very useful for reporting, hard to add the same test case in multiple suites without copy-pasting, issues when integrating with JIRA

Vorteile

This software has been key in our regression testing. This helps us make sure our applications core features continue working as we add features. The API is great to use to for our automated testing.

Nachteile

There is a learning curve on the terminology I feel. Test Runs And Test Suites and Test Plans etc. It makes sense after a while but takes some getting used to.

Vorteile

Valuable reports (that's the main purpose of testing)
Easy to onboard
Supports is repsonsive and helps with your case
Good API and integration with JIRA

Nachteile

There is no integration with confluence (to get bubbles like for JIRA defects/tasks)
You can't push an attachment in the jira (only can show an attachment in jira's plugin)
API needs some extra methods to work with plans and runs with configuration (you allow to manipulate with it, but only in strict cases)

Sometimes it takes too muck time to get a reply from staff on the forum.

Vorteile

I love the integration with Jira and Ranorex. It makes the whole test execution process seamless.

Nachteile

Probably more of a process issue but I find that we can copy tests all over the place and start to lose our version control.

Vorteile

- managed in cloud
- powerfull customization options
- easy to use
- option of templates for TC

Nachteile

- some reports are complicated to get, as who worked on what in the past quarter
- to customize some stuff is sometimes quirky
- hard to use for exploratory testing
- collaboration on preparing test plan is a pain, it does not allow two testers add stuff to test plan simultaneously

Vorteile

Good way to organize test plans, cases and runs
Report is good at test planning level
Configuring runs for multiple devices/browsers in run configurations instead of creating new runs
APIs are good,
Can be integrated into our build process well.

Nachteile

nothing much. may you can support hippa complaint could offering?

Vorteile

The ability to go as deep or shallow as you want
Integration to automated tests
API and CLI functionality

Nachteile

Looks dated
Can be hard to find/navigate some tests