Quality Control for User-Submitted Content
Niche employs a number of quality control measures when incorporating self-reported user data and user reviews. While there is no 100% foolproof way to ensure that one specific review or piece of data is accurate, these techniques are meant to 1) minimize the impact of potentially inaccurate or erroneous data and 2) ensure that data at an aggregate level is representative for a given school.
Submission of User Content
User-generated content on Niche is collected via our school surveys and account signup, which you can access in the navigation menu at the top of any page and from links throughout the site. In order to submit user-generated content (including survey and reviews), users must register for a Niche account. During registration we collect information that allows us to identify potentially fraudulent users/reviews that may not be associated with the school. Note that we do not require a school affiliated email address, which some schools have asked about. This is because 1) not all schools provide consistent school affiliated email addresses and 2) we want alumni to be able to review their school and many alumni either do not have access to or do not regularly use a school affiliated email address.
Quality Control Procedures
We have three main phases of quality control with respect to self-reported user data and user reviews.
1. Automatic quality control measures
These attempt to automatically prevent both erroneous data (from user error) and fraudulent data from ever making it to our website or impacting scores. These look for data outliers (i.e. born in 1900) as well as inappropriate and/or unhelpful words and phrases (swear words, “I don’t know”, etc.)
2. Ongoing community flagging/feedback
There is a “Report” link next to all user generated content on Niche that allows any Niche visitor to submit any content as inaccurate or offensive which then is manually reviewed by a Niche employee.
3. Regular data updates/audits
When scoring user reviews or user data for rankings and on-site features, we have additional automated scripts that account for anomalies and suspicious activity – things like several 1 or 5 star reviews in a row, multiple reviews from the same IP address, or a user’s address location not matching the school’s location and characteristics. We also employ a number of statistical techniques to reduce the impact that a small number of very happy or very unhappy users can have.
Quality control is a constantly evolving process at Niche that we strive to continuously improve as we get more feedback from our users, schools, and the data itself.