by Elizabeth Irish, MLS
Autism Asperger’s Digest| March/April 2012
In Part 1 of this series, we explored effectively designing a search strategy and selecting sources. I hope you were able to uncover some solid information that helps you be better equipped as a part your child’s healthcare team.
Evaluating Web-based health information is like solving a mystery. How do you know that information is reliable? One article recommends a certain approach; another recommends something completely different! There’s so much conflicting information on autism spectrum disorders: what’s trustworthy, what’s too good to be true, and what is a bit of both? A searcher’s clues to reliability are up-to-date information, authority, bias, and content. What clues do you look for to uncover quality? Are the author’s motives pure, or is there a hidden agenda for you to buy a product? So how do you find reliable, quality information?
Establish Time Frame
When reviewing a website, establishing when the material was written and last reviewed is equally important. In healthcare up-to-date information is vital, especially with therapies, treatments, and research. This can be overrated though; basic anatomy hasn’t changed in years! A good way to determine the age of a website is to find a publication or copyright date on the site. Some health sites list the date the page was created, reviewed, and last updated. If a site doesn’t have a date on it, try another to see if the material is the most current.
Identify Suspects and Motives
The first chapters of a good mystery identify suspects and establish motives. Does the suspect have the knowledge, background, and connections to have committed the crime? Why did the crime occur? What did the suspect stand to gain or lose? The same questions should be asked when reviewing information resources. Authority and bias are key factors in determining quality.
Authority: Does the suspect have the knowledge, background, and connections to have committed the crime?
Does the author have the education, credentials, or experience to back up the work? A health website takes on a completely different flavor if it is written by a healthcare professional, a patient, a parent, or an advocate. If you are researching information on a diagnosis, therapy, or prognosis to make a healthcare decision, then a site developed by a healthcare professional with the appropriate educational background and experience is a site to consider. However, a first-hand account from a patient or parent is better to connect with someone in a similar situation. Social and personal accounts fill as great a need by providing comfort, life experience, and a sense of community. Web 2.0 resources, such as blogs and other social networking tools, can play a huge role. As long as you know who is behind the statements, you can sift through the information accordingly.
However, even a title can be misleading. A “doctor” can be an MD or a PhD, perhaps in an unrelated field. Does the person have updated credentials? If you’re suspicious here are a few places to check:
- American Board of Medical Specialties
- American Medical Association’s Doctor Finder
- State Department of Health websites
Another authority clue can be found in the About link. A website should be transparent about its purpose and origin. About normally contains information on sponsoring organizations, institutions, or authors. If it’s a large site with numerous articles, each article might have the author’s credentials on the article page. Is there an editorial board, and is the editorial policy easy to find? If no one is willing to take responsibility for the content, think twice about its advice!
Contact Us links also provide clues. Nothing is as frustrating as hunting for an email or contact number! I get suspicious when I have to spend more time searching for contact information than I do searching the site.
Finally, the site’s domain offers clues. The site domain is the last two or three letters of the website’s address. For example, a site ending in .edu is a good indication that it’s from an academic institution. Some common domains are the following:
Commercial = .com
Government, U.S. = .gov
Society or organization = .org
Network, or Generic = .net
International sites = two letters
(e.g., .ca [Canada], .uk [United Kingdom])
A word of caution: not every country follows the same regulations for a disease or condition. Prescription drugs, over-the-counter drugs, herbal remedies, and supplements can vary by country. What you read might not be applicable or available in your country.
Bias: Why did the crime occur?
If determining who authors the resource is difficult, it’s possible you’re dealing with bias. Why was the website created? Why was the article written? There’s always a reason. Is it to share knowledge or sell a product? Let’s face it—everyone has motives, but we need to determine if they’re altruistic, unintentionally misleading, or even nefarious.
Some motives are overt; others are covert. Overt bias is easier to identify; as long as it’s recognizable, you can make up your mind accordingly. Drug studies sponsored in part by a drug manufacturer aren’t necessarily bad as long as you know. Many publications and sites have authors sign disclaimer forms, letting the audience know of any relationships that may impact their research, theories, or views. This disclaimer should be easy to find. The efficacy of natural medicines is another good example of where strong emotions may come into play. If you know where the authors stand, you can take their position into account as you examine the content. There may be a motive, but it’s upfront.
Bias: What did the suspect stand to gain or lose?
Covert bias can be more difficult to discover because it can present skewed information in a plausible format. Is the website a cleverly disguised advertisement? Ads can be presented as legitimate articles to sell a product. Are site ads clearly identified as advertisements, or do ads look like part of the content? Sometimes it’s funny to see an ad labeled, “This is an advertisement,” but in reality it’s a sign of good faith.
In line with motive be wary of any site that asks for personal information. Don’t give up private information if you’re unsure of a website! Never give up your passwords. Check out the privacy and security statements to make sure you won’t become a victim.
In looking at authority and bias, the best advice is to always ask: who and why? The answer is telling. You may decide to stop your investigation at this point, but if the clues are good, now it’s time to look at the evidence.
Examine the Evidence
What is the Web content based on? Even if it’s based on a study, there’s still a mystery to be solved! How often have you heard a statement begin, “Well, the research supports….” OK, but what does the research actually say? Statistics can mean anything, depending on how they are presented! The following section isn’t meant to replace a research methods course that spends hours on this premise, but here are some helpful questions to ask during this phase of the investigation:
Content: Can the results be reproduced?
Just as a detective interviews more than one witness, always try to consult more than one resource. The results of an experiment should be reproducible. Don’t hesitate to do a PubMed or Google Scholar search to look for supporting articles. While it is true in certain circumstances that a treatment may be so new there is little evidence in the literature, there usually is at least one study. Often in medical articles the original study is mentioned in the article. If there isn’t a reference, but the author(s) and institution(s) are listed, then try searching Google or PubMed with that information. Ask yourself:
- Are references to other sources of information clearly provided when appropriate?
- If not can the information be verified in other resources?
Content: Is the study design valid?
Study design makes or breaks a claim. A few things to consider when looking at a study:
- Study size—Were there enough participants to make the results statistically significant (i.e., the results couldn’t have happened by chance or error)?
- Inclusion/exclusion criteria—What criteria did the researchers use in deciding patient participation?
- Randomized—Were patients put into an intervention group (i.e., treatment vs. placebo) using a randomized process before the study began?
- Confounding factors—Is there a variable that applies to both interventions that may skew the results?
- Blinding—In a double-blind study, neither the patients nor the researchers know which patient is receiving which treatment.
- Numbers needed to treat (NNT)—This figure calculates how many patients receive the treatment before any good is done.
Content: What is the level of evidence?
Some websites provide levels of evidence. They are often found on professional healthcare websites to give healthcare providers an idea of how sound the research is behind the study. There usually are no more than five levels. The criteria used to establish the levels should be available either on the same page or through a link. Always select a study with the highest level of evidence!
What’s the basis for levels of evidence? As a rule of thumb, the highest level of evidence is filtered information such as systematic reviews or critically appraised topics. These have standardized methods of reviewing research for quality. The next level of evidence would be unfiltered information such as a stand-alone research article. In this level look first for randomized controlled trials, followed by cohort and case studies, and then editorials or other expert opinions. Keep in mind that not all study types fit all circumstances, so a higher level of evidence may not be available.
MedlinePlus is an example of a free website that links to other reliable sites. Developed by the U.S. National Library of Medicine, National Institute of Health, it’s a good central starting place to find information from government, education, and organizational sources.
Health Topics breaks down topics by body location/systems, disorders and conditions, diagnosis and therapy, demographic groups, and health and wellness. A definition is provided for each topic, then each topic is further broken down into subtopics such as overviews, research, and treatments. The subtopics provide links to government, academic, and organizational sites.
Drugs and Supplements provides information on prescription, over-the-counter (OTC) drugs, herbal remedies, and supplements. Prescription and OTC medication information is from the AHFS® Consumer Medication Information, produced by the American Society of Health-System Pharmacists. Herbal and supplement information is from the Natural Medicines Comprehensive Database Consumer Version, Therapeutic Research Faculty.
The Videos and Cool Tools section includes videos on surgical procedures, news items, and anatomy. There are interactive quizzes and games to help test your healthcare knowledge. Links to health calculators and checklists are also available. For instance, for autism there is a link to the Centers for Disease Control (CDC)’s Developmental Milestone site.
MedlinePlus is a continually evolving site that is definitely worth a look. It’s a good place to start for general as well as more specific information from government, education, and organizational sources.
Health on the Net Foundation (HON) is a non-profit, non-governmental organization that is accredited by the Economic and Social Council of the United Nations. Based in Geneva, Switzerland, its mission is to protect people from misleading health information. Website developers can request that their site undergo a rigorous evaluation to receive HONcode certification. This allows them to post a HONcode icon on their site as a symbol that the site follows ethical principles, which include authority, justifiability, transparency, and financial disclosure. A HONcode certificated site receives check-up visits and must be recertified at a scheduled time to ensure continued compliance. HON will also investigate if they receive a complaint about a site. To date over 7,300 sites have been accredited in over 100 countries.
In addition to certifying sites, HON provides individuals, medical professionals, and Web publishers with searching options. There is also a feature called HONsearch, which allows you to search for HON certified sites. HONtopics is similar, although in this case the topics are automatic searches based on pre-formulated topics.
Given the purpose of the site, there is a great deal on Web evaluation and how HON certification is achieved. Seeing the HON icon can provide peace of mind that the site’s developers cared to take the time to be certified.
Evaluating Health Information on the Web: What Other Sites Say
Here are a few sites to support the recommendations within the article. It is best to try to validate what you read in numerous sources!
Medical Library Association. A User’s Guide to Finding and Evaluating Health Information on the Web.
The Medical Library Association (MLA) is the national organization for health information professionals. In addition to a content evaluation guideline for health information, the guide also includes a top 10 list of consumer health websites, plus additional recommended lists for cancer, diabetes, and heart disease websites.
U.S. National Library of Medicine. National Institutes of Health. MedlinePlus: Evaluating Health Information.
Brought to you by the National Library of Medicine (NLM), this site provides a basic guide to evaluating health websites as well as links to other evaluation websites such as the Medical Library Association (MLA), the National Cancer Institute, National Center for Complementary and Alternative Medicine, and the American Academy of Family Physicians. Other links send you to basic overviews, recent news items, finding information on specific conditions, and related issues. There is also a 16-minute tutorial on evaluating health information for those who wish to dig a little deeper.
National Center for Complementary and Alternative Medicine. NCCAM Basics. Evaluating Web-Based Health Resources.
NCCAM’s recommended approach to evaluating natural medicine websites. Provides a list of additional resources and a link to their Time to Talk campaign. Time to Talk provides tips on talking with your healthcare provider about CAM therapies.
Wrapping Up the Case
Like a detective you should be an impartial observer when evaluating health information on the Web. Finding up-to-date information, authority, bias, and content can help take the mystery out of website evaluation. Remember that the Web is full of information, but a good detective can separate the good from the bad!
Elizabeth Irish, MLS, AHIP, is Assistant Director at Schaffer Library of Health Science, Albany Medical College in Albany, New York. Her son has the dual diagnosis of AS and Tourette’s.
Copyright © Autism Asperger’s Digest. 2012. All Rights Reserved. Distribution via print is prohibited without written permission of publisher.