Product Doc: Ask

What it is

‘Ask’ enables editors to create embeddable forms to invite contributions from readers. These could come in several formats, including text, photo, video, audio. The contributions can be (optionally) linked to existing user profiles. Editors can filter, sort, share, and manage the contributions, and then display the best ones in a gallery.

(NB the traditional term for this is UGC; however, we don’t want to encourage the separation of ‘comments’ and ‘ugc’ as terms – after all, comments are user-generated contributions themselves. The principle differences here are in application and expectation: end users don’t expect their contribution will necessarily be displayed, and contributions are always many-to-one, never many-to-many.)

Who is it for?

  1. Engagement Editors and Journalists – people who invite, manage, and publish user-generated content in order to
  • enhance their stories
  • find information to improve their reporting
  • maintain institutional memory about high-value contributors
  • have complete control over the look, feel, and functionality of call-out forms
  1. End users – people who consume content and want to contribute personal knowledge and experiences, in order to
  • improve the quality of the coverage, so that it better reflects their own needs/experiences
  • gain recognition from a relatively high-status voice in their world (the journalist) so that they can feel validation for their knowledge/experience, and then share that recognition within their social circles
  • help a journalistic publication to which they feel attached, so that they can feel a closer part of its mission to inform, educate, entertain
  • feel acknowledged and remembered over a history of contributing to the organization in different ways
  1. Publishers – people who want to
  • know more about who is contributing to the org’s journalism
  • better understand the value of investing in people and tools
  • avoid sharing critical, potentially sensitive user data with third-party, cloud-based systems

Glossary

Call out – a request for information from end users.

Moderation – acting upon contributions based on quality as measured through various metrics, including potential usefulness for the journalism; offensiveness/breaches of on-site guidelines; breaches of copyright.

Typology of Crowdsourced interactions

Taken from Tow Center report on crowdsourcing in journalism

  1. Voting—prioritizing which stories reporters should tackle.
  2. Witnessing—sharing what you saw during a breaking news event or natural catastrophe.
  3. Sharing personal experiences—divulging what you know about your life experience. “Tell us something you know that we don’t know.”
  4. Tapping specialized expertise—contributing data or unique knowledge. “We know you know stuff. Tell us the specifics of what you know.” [including: can you help us verify this?]
  5. Completing a task—volunteering time or skills to help create a news story.
  6. Engaging audiences—joining in call-outs that range from informative to playful.

Also, not in the report, crowdsourcing after publication:

  1.  Reporting corrections / clarifications—improving the story’s factual/grammatical content.  

 

Types of media

In order of priority for our build

Version 1:

Text

Future versions:

Photo
Video
Audio

Sources of contributions

Version 1:
Embeddable or standalone forms build through our tool

Future versions:

SMS
Social: Instagram, Twitter, Facebook, Snapchat
Messaging apps: Telegram, WhatsApp, Kik, Facebook Messenger

 

Why build it?

  • It matches core needs (see below)
  • Existing tools aren’t optimized for journalistic use – common complaints center around poor design, poor mobile experience, poor accessibility, time consuming to create galleries/share the results based on call outs, inability for contributors to be remembered across different call outs, don’t connect with existing user databases or include the context of previous comments/contributions
  • It’s an important part of our “contributions, not just comments” philosophy
  • It allows us to build and test scalable curation management tools with something more manageable than a full comments system, as Ask isn’t real time or user-to-user facing, so likely to be lower volume and suffer less trolling

What are the core needs this product will address?

 

Engagement Editor/Journalist needs:

  • To be able to identify/invite knowledge from within the userbase, via an easy-to-use photo/video/audio/text upload tool, so that the journalists can find more stories/sources.
  • To be able to identify and display the best contributions quickly and easily, so that users’ knowledge can be found and integrated in the narrative in the period when people are still visiting the story for the first time, thereby encouraging larger readership and more submissions.
  • An interface that is fast and easy to use, so that it doesn’t add too much strain onto the existing workflow.
  • An understanding of the expertise and interests of the most engaged readers, so that the journalism can be more targeted to people’s needs.
  • Ability to ensure anonymity of sources when required.
  • To be able to verify quickly if something is genuine, and if it came from the person who claims it,  so that inaccurate information isn’t shared.
  • To be able to know if a user has a history of reliable or unreliable contributions, so that they can put the contributions into a wider context.
  • A way to understand which call outs have been successful and which have not within various contexts (averages for the site, for the section, for the topic, for the medium, etc), so that future call outs can yield better results. (Analytics)
  • Proper spam/troll filtering, ability to avoid bot swarm so that signal/noise ratio is high.
  • Ability to share moderation across many people so that a popular call out can be managed.
  • Ability to embed anywhere in the article so that it isn’t only at the bottom of each article, where fewer users will see it.
  • Ability to link directly to the call-out box within the article from social media, so that people can be directed straight to it and don’t have to scroll down an article to find it.
  • Ability to contact the user (with their explicit opt in) if/when another UGC request/story on this topic is published so that momentum and interest around the topic are not lost.
  • Ability for user to send only to the journalist, not to have it shared publicly.
  • Ability to see quickly when a user’s contributions have already been featured in a recent call out so that a wide range of users are featured on the site. Also ability to see which contributions were used where.
  • Ability to connect with existing tools so that existing workflows don’t have to change, and newsrooms have access to the widest range of options.
  • Ability to add tags and notes to contributions, both on the curation management side and also end-user facing, so that context and knowledge can be added.
  • Ability to group and filter contributions, both on the curation management side and also end-user facing, so that patterns can easily be identified and shared.
  • Ability to allow users who are logged in to create pseudonymous/anonymous identities for each individual contribution, so that persistent knowledge of users can be learned without having to put their identity to every contribution.
  • Ability to help journalists learn who are those engaged users on a topic so that they can see the value of their audience.
  • Ability to make calls to learn more about the users to feed data straight into Trust so that we can reach out in the future.
  • Ability to mask potentially offensive content in the moderation queue.

 

Developers need:

  • A way to turn information from a form into something they can process on their own servers, not on a Google spreadsheet, so that end users who are wary of Google storing their data (especially in Germany) don’t have to have their information stored externally.
  • A way to connect their existing user database to the user information learned here.
  • Software that’s compatible with their existing site architecture. (Easily importable/embeddable.)
  • Ability to tweak the software to add/delete features and elements as needed by the org.
  • Software that adds only minimally to page load.
  • Software that is secure: doesn’t allow calls from outside a structured set to trigger database actions.

 

Publishers need:

  • A way to measure if deeper engagement leads to subscriptions/other type of monetary or other commitment with the company so that they can put a clearer monetary value on engagement.

Users need:

  • A way to log in to connect the contribution to previous contributions (opt in), so that they can benefit from good reputation.
  • Adequate error messaging to explain common user problems. 
  • A way to understand clearly how their content will be used and distributed to other platforms.
  • A way to share their experiences in a way that they feel listened to, so that their views or experiences are reflected more in the coverage; because they want to change editorial choices; because they want to contribute a story to their favorite news outlet.
  • A way to share corrections with a sense that they are being seen/listened to so that errors can be corrected quickly.
  • To know how their information is going to be used, so that they aren’t concerned about a private tip being publicly displayed against their wishes; so that they know if they can keep the copyright; so that their names aren’t being publicly attached to private information, so that they can feel they can trust the news org.
  • To be able to seamlessly contribute from any internet-enabled device. 
  • To know if their contribution has been used in a meaningful way – either to update/improve the story, or actually published within it, so that they feel their effort has been worthwhile and appreciated (How can we automate this?) 
  • A way to view, export, remove their contributions from the system so that they can have a record of their work, and also control its usage.
  • A way to have their inputs automatically saved  locally before they submit the form in case they accidentally lose the data / lose their connection before submitting/want to return to it later; then that data being deleted on successful submission so that their privacy is intact.
  • A way to like each submission so that I can acknowledge someone else’s contribution – similar to comment liking, flagging, recommending, thanking.
  • A way to link to each contribution individually

 

What does it contain?

Embeddable elements for the article page.

They must:

  • be responsive
    • mobile
    • desktop
  • not substantially add to the page load or render times
  • be compatible with standard in-app usage
  • be secure
  • have login options
  • include the ability to add login/captcha after a call out has gone live if a particular call out’s contributions become filled with noise/spam/harassment
  • Be able to handle DDOS attacks/bots

Types of embeddable call outs/display:

  • Forms
    • Form builder
    • Form embeddable
    • text box
    • radio buttons
    • dropdowns – accordion and others
    • inspirations: Typeform, Hearken, Google Forms, Zurb’s Foundation, Sub at WaPo, Formacist at NYT
  • Voting / Polling tool – with options to keep private, make public automatically, surface only after closing poll
  • Upload media
    • photographs
      • multi-uploader / single upload
      • captioning
      • basic pre-upload editing?
    • video
    • audio
  • Integrations with device-capturing capabilities (mobile phone cameras, etc…)

An embeddable display unit

  • Showcases the best content that has come in from the call outs.
  • Each individual contribution has its own URL for sharing/embedding.

Curation views

  • “Create an Ask”: Builder for embeddable elements
    • simple UI
    • ability to preview elements
    • embed code generator for CMS
    • ability to add Captcha
  • “Ask List”: Index view of all the different call outs created
    • listed chronologically, with counts of contributions/flags
    • ability to filter by section, date, type, currently active
    • Contains free search within submitted data, meta data, users
  • “Curation”: Index view for each call out
    • lists all contributions chronologically
    • links to/shows the live embeddable element and related data such as:
      • creator
      • timestamp when created/made live
      • timestamp when placed on page
      • timestamp/s when changed
      • timestamp when no longer live
      • article url/s that it’s used on
    • ability to filter contributions by different fields
      • username
      • type
      • flagged/not flagged
      • approved/not approved
      • tags
      • displayed/not displayed to readers
      • marked as useful/not useful by the journalist
      • Also contains free search
  • Moderation view of unmoderated content
    • keyboard shortcuts
    • ability to select multiple entries
    • ability to zoom in on one entry
    • ability to see more information about the contributing user
    • ability to filter by “users whose work has been used before” / “users who have been warned before for inappropriate content”
    • ability to write notes
    • ability to add tags
    • numerical count of to-be-moderated contributions in this call out
    • allows multiple users to work on this view concurrently
  • User-list view
    • filter by name
    • filter by number of contributions
    • ability to write Trust-style formulas to create lists
  • Spam/abuse management view
    • contains customizable word filters with easy import/export of lists
    • lists of removed contributions to review for possible false positives

User views

  • Basic profile view showing all of a logged in-user’s contributions
    • ability to download photo/audio/video at original quality level
    • ability to delete contributions
      • NB if a contribution has been published as part of a story, a user might need to fill in a separate form to request removal

Also needed

  • Some form of clear training/help on how to use the system

 

How will we measure if it is a success?

  • Surveys of frequent, occasional, and non-users of all kinds
    • engagement editors
    • end users
  • Track usage by engagement editors on current system vs new one
    • select a few basic tasks
    • time how long it takes frequent users to do them on both systems
  • Track number of views of contribution display unit
  • Track incomplete forms vs complete
    • including number of people put off by need to log in
    • try with and without login?
  • Track number of queries on index views per minute
    • a lot might indicate higher usage, or inability to find what the engagement editor wants
  • A/B tests
    • positions on page
    • style of call out language
  • Track the number of contributors and contributions on current system vs new one on similar topics
    • taking into account if they were publicized on social/internal ads, featured on main page, written by prominent journalist, etc
  • Track return visits to a page
    • aggregate and possibility indiv contributors
  • Track visits to and interactions on user profiles
  • Track number of messages sent out by journalists
    • Also the number of responses

Current state of Crowdsourcing in newsrooms

The Tow Journalism Center published a report on crowdsourcing in newsrooms in December 2015 (which mentions us). This is required reading for the team before we start building: http://towcenter.org/research/guide-to-crowdsourcing/

Future iterations

  • Message view
    • ability to send template or personal emails to single or groups of users who opted in to be contacted
    • ability to automatically email users when their contribution is included in an embeddable display unit
      • ideally with a URL directly to their contribution
    • ability to see previous messages sent and by whom
      • possibly only marking which journalist sent them a message, not the content, for greater privacy
      • replies come straight back into this view as well as the journalist’s email
      • allow journalists to reply via their own email client as well as through the system
        • include their email replies in the message view too
  • Notifications for engagement editors when certain thresholds of contributions are reached
  • Connections between the Identities of end users who submit across multiple forms
  • Connections between the identities of end users and the site’s pre-existing login (opt in)
  • Different login levels
    • hiding PIIs from some levels of auth
    • access only to certain defined call out moderation queues
    • restrictions on who can add content to public-facing display
  • Quizzes similar to Playbuzz
  • Crowdsourcing tools that allow for multiple-consensus confirmation
  • Badges to reward contributors, displayed publicly
  • Tracking of abnormal behavior, leading to notifications
  • Detailed analytics around interactions and engagement for engagement editors
  • Analytics for users around usage of their work
  • Ability to connect with basic image verification tools, eg. Verified Pixel, First Draft News
  • Predictive analysis of wording of forms/other elements for optimal levels of response
  • Predictive estimates of amount of time to sort through the incoming queue/amount of data storage needed
  • Ability for users to request emails/notifications around similar topics/call outs
  • Ability to automatically generate lists of users who are likely to be interested in contributing to a particular call out
    • Based on topic/media type/author
  • Ability to feed in data from third party apps/sources
    • Hearken
    • Screendoor
    • Groundsource
    • Google Sheets
    • Excel spreadsheet
    • CSV
    • Twitter cards
    • Telegram
    • Messenger
    • Instagram
  • Ability to connect Call Out database to subscriber database
  • Multi-leveled permissions – Not everyone should be able to see confidential information that people have supplied only for certain journalists to access. Also ability to add moderator access onto to a particular Ask, not all or nothing.
  • Ways to plug in automatic verification for images/videos, eg Clarifai
  • Autosave on form builder, with some version control (eg. autosave every 10 minutes)
  • Ability to crowdsource other tasks
  • Ability to create Asks that require unique codes in order to respond, to limit respondents to only certain chosen subsets of people
  • Anti-spam, ability to add RECAPTCHA
  • Ability to create Asks on mobile
  • Version designed specifically for handling Corrections/Clarifications on sites
  • Keyboard shortcuts for moderation/creation
  • More detailed comparative analytics between call outs
  • Export as CSV/Excel
  • Ability to like/thank other people’s submissions
  • Ability to create admin alerts – eg. an org in the US is legally liable if someone edits a contributor’s content and materially changes the meaning
  • Conditional logic (e.g.: IF response from a multiple choice field is B, show these B related questions below)

 

Basic competitive analysis

Screendoor

From the Tow Report: “[Screendoor] allows ProPublica to filter and rate responses, add comments into the responses, send a note to a specific sub-group of people, track the emails sent, and share all project updates with participants in a personalized way. “I can say, ‘Dear Barry,”’ Ornstein said. “You told us you have cancer, diabetes, or [suffered from] a heart attack.” Also see this presentation: How ProPublica uses ScreenDoor

The Counted from The Guardian

In particular, its tip forms and how it displays information

N0tice from The Guardian

Forms the basis for the previous version of their Guardian Witness app

Typeform

Google Forms

http://www.getchute.com/

Also see: https://community.coralproject.net/t/a-survey-of-existing-user-generated-content-tools

Potential UI Inspiration

Splashthat.com

Picatic.com

Kinja’s simplicity esp with formatting tools

Instagram, Twitter, etc.

Public Insight Network

Audience Engine

Checkdesk

Verification Junkie

Tumblr

 

Sample Scenario

In the daily meeting at ThisIsTheNews.com, Kerry hears Latoya, a business journalist, say she is working on a story about workplace harassment law. Kerry approaches her after the meeting and asks if she has considered making this a series of pieces, featuring people’s experiences of workplace harassment. Latoya and Kerry together pitch this to the editor, who approves the idea.

This will be a four-part series that shows the complexities and realities of dealing with workplace harassment, and the different legal steps that can be involved in taking action around it. Latoya and Kerry together figure out what success looks like for this series. They decide that success will be a series that:

  • helps people understand the legal system with regards to harassment, including its flaws
  • helps those currently experiencing harassment to understand their options, and feel less alone
  • helps people who are experiencing, or have experienced, harassment to feel listened to, and to feel that their experience could help others
  • shows the diversity of experiences related to this topic
  • highlights flaws in the system for current lawmakers to consider

Kerry and Latoya decide to create a series of call-outs at different stages of the series, asking for specific information from readers who have personal knowledge or experience of the subject.

The first article is an overview of the current situation, with three anonymous sources describing their experiences. At the halfway point of the article, they decide to put a form that specifically invites more stories similar to those in the piece.

At the top of the form they write a short paragraph explaining what they are asking for, and how it will be used, as well as promising anonymity.

The form they create invites readers to:

  • share details of their story
  • choose whether their story is only for the journalist or for publication
  • choose between leaving their personal information or being anonymous
  • attach their comment to their existing login
  • list the date / period that the incident/s occurred
  • name the location/s where the reader was working
  • share contact information with the journalist
  • answer a few specific questions (eg “What do you wish you’d known about the process?” “What would you say to someone else going through what you did?” “Did this have a significant financial impact on you and your family? If so, how?” “Did you contact a lawyer?”)

Latoya creates a first draft of the form, and shares it with Kerry via a URL that is only viewable by someone logged into the newsroom system. Kerry suggests some edits. When they’re both happy, Latoya clicks “Publish”, and then pastes the resulting embed code into the article page on the CMS.

Latoya finishes her article, submits it to the editor who makes a few small changes then publishes it on the site. Once the article is published, a note is automatically added to the Created Form Page noting the page URL where the form is embedded.

The form stands out from the text but matches the article design, so it doesn’t look like an ad. It is equally easy to read and enter information on mobile and desktop, and if someone loses their connection, the information they entered does not disappear from the form. Autocomplete is auto-disabled for all fields, so that any private information doesn’t then appear if someone else starts filling out a form on the same computer.

Kerry shares the article with non-profit and legal groups that work on this topic, and asks them to share it with their communities. She also shares it on social media.

After half a day, Kerry and Latoya log in to the system. They want to see how many responses they have received, along with some context about when the responses came in. There are a few abusive messages; they quickly flag those so that the system has a record of them, and they don’t show up in their future searches of the submissions.

Latoya first wants to read the notes submitted only to the journalist, sorted between those who left contact information, and those who did not. She reads each complete submission of those who entered information in that field, and contacts people who left the most compelling stories, in order to gather sources and verify facts for her follow-up stories. She also asks them what information would be most useful for people in their situation, in order to move closer to the definition of success for the series.

She then filters the responses by date that the incident/s occurred, and only within her city.

There are so many good responses. Latoya decides to create a new story that has several of them displayed within the text. Kerry also advises her to share a small number of the best entries directly next to the form, in order to elicit more submissions.

Latoya first selects small selections of content to place in a gallery alongside the form. She chooses the entries she wants to include, then selects “create gallery.” She edits some of them for clarity, concision, and to remove any identifiable information. These small selections contain a few lines, and the first name and age of the respondent (taken from the form, with those fields selected by Latoya). An automated message displays on the page next to the content: “Some of this content may have been edited for clarity and/or legal reasons” Her edits are placed in a new field alongside the original submission.

At one point, she accidentally tries to add a piece of content that was not approved for publication by the user, who ticked the box saying “this is only for the journalist, not for publication”. The system serves up an error message, preventing her from doing this.

Latoya clicks “Preview” to see how these responses look in the gallery, both on desktop and mobile. When she is happy with the results, she hits “publish”, and names the gallery for internal use, “Office Harassment 1.” An embed code is then created, which she pastes next to the call-out, in order to encourage more submissions. Within the gallery, each published piece of content in this gallery has a URL to go directly to that submission. She uses the system to email the URLs to each person who submitted an answer that was used, and who left a contact email address, to tell them that their story has been included.

After four days, Latoya decides to move on to the next piece in her series. She goes into the system and closes the form to submissions. She types a message to appear in its place: “This submission form is now closed. If you have a story you’d like to share, please email me at latoya@thisisthenews.com” The form now is closed, and displays only her message and her gallery.

 

——

 

Alice is a regular commenter on ThisIsTheNews.com. She sees the article on workplace harassment. This is something she has suffered, but she has never told anyone about this outside of her close family.  She reads the piece, and feels that her perspective isn’t reflected by those in the article. She reads the gallery submissions, and feels that she has more to offer that isn’t included yet in the story. She sees the form, and hesitates. She doesn’t want people who know her, to know what was done to her.

She reads the form information. It makes her feel more confident that she can share her story without giving up her identity. She also believes that the journalist will read what she says (unlike the comments she leaves on the site.)

She types in her story. The system has her logged in as a commenter, but that is clearly identified in the form, and in order to connect her on-site identity, she has to click a box. She reads why the organization wants to connect her comment ID, and understands their point of view, but decides not to do so. She leaves the same email address as she used to create her commenter ID, but the system does not make the connection. She adds the city where she lives, and taps “Send”. An automated message thanks her for her contribution.

The next day, she receives an email saying that her story has been included in a gallery of stories. She clicks the link, concerned that her identity might somehow be compromised. The link goes straight to an edited version of her words in the gallery of submissions. It simply reads “Anonymous, Chicago.” She breathes a sigh of relief, and reads the other submissions. She feels that she trusted the news org, and that they respected that trust. She hopes that her story helps others. She scrolls down to read if any comments make reference to her story. Also in the email was a link to subscribe to notifications for when the rest of the series is published. She goes back to the email, and clicks to subscribe.

———