Draft Proposal: Addendum to analytics proposal

[BeDiggy]

This proposal, is actually an Addendum to the passed analytics proposal. The goal is to make the “Alpha” version of the open-source ShapeShift web app with analytics, the default web application.

PROPOSAL DESCRIPTION

Summary:

There is an extreme need for analytics in the product that we’re building, and the ability to understand how we’re doing in our goals as a DAO, if we are building the right features, and reaching product market fit. After discussion on product and engineering and with the community and other members in the DAO, this proposal was created as an addendum to the original proposal.

This addendum is to make the “Alpha” version of the ShapeShift web app the default application that users are directed to. So, the URL for the analytics experience would be app.shapeshift.com.

It will still be an opt-in version that users will agree to participate in upon entering (as voted on and approved by the community and token holders in the original proposal). If the user does not wish to opt-in, they can choose to go to the private version that will have no analytics. See mock up for the idea—please note that the copy is not final:

This proposal is simply to clarify that app.shapeshift.com would be the opt-in default application with analytics, and private.shapeshift.com will be the optional application with no analytics.

Motivation:

We have a responsibility to the DAO and the FOX Token holders. How do we prove and measure the effectiveness of what we’re doing? Are we successful? Are we reaching product market fit? Which marketing messages and programs lead to spikes in user acquisition? Are we retaining users? We require a large portion of our user base to use the version with analytics to get the data we need to be able to answer these questions, and to draw insights on where we’re succeeding and where we need to improve. Therefore, the Product & Creative Services Workstream is proposing to have analytics as part of the default web experience.

Specification

Alpha & Private Versions

With alpha becoming app.shapeshift.com, it would no longer be a step in the CI/CD pipeline. It will remain as we currently have it. As features are launched and tested, they are pushed to app.shapeshift.com behind feature flags. Once Ops, Product and Engineering are confident that bugs are eliminated and the feature is working as expected, the feature flags will be removed. The feature would then be rolled out to private.shapeshift.com.

Maintenance

Nothing changes from the original proposal.

Reviewing Data

Nothing changes from the original proposal.

Budget

No budget is required for this addendum.

Benefits

Reiterated from the original proposal:

The Alpha version will give us the analytics and user input we need to make the product awesome before pushing it to the Private version.

By using Pendo, we’ll have an all-in-one platform for tracking, analyzing, and creating guided user journeys (instead of paying for multiple platforms). We expect the initial setup to take several weeks to get all the guides and the resource center up and running. As mentioned above, all of that will be handled as part of the Product Workstream responsibilities. It will not require engineering efforts.

Retention is an important metric, and one that can be improved by utilizing this approach. We’ll be able to take our user experience to the next level, guiding and engaging our users at the right moments, leading to better feature engagement and higher user retention. Without the metrics we would have an extremely difficult time determining where to improve the UX and measuring the success of our product.

New feature adoption rates will also be evaluated as a metric to measure success. We will be able to see which marketing messages, promotions, and tactics are the most effective in driving users to the app.

Drawbacks

We will still have both versions, as outlined in the original proposal. One drawback could be that if we don’t get enough users opting into the analytics version, we would still be struggling to get the valuable data that we need.

Vote

If you vote “for” this proposal you agree to have app.shapeshift.com the default (opt-in) version with analytics.

If you vote “against” this proposal, you do not think this is the correct approach

[MrNerdHair]

“Default opt-in experience” is an oxymoron, and I don’t support analytics by default. That said, I do support the user making an affirmative choice one way or the other, which would mean that neither experience would be the default.

I don’t think specific implementation details like the setup of CI/CD should be part of this proposal; likewise, we should not specify in detail that feature flags must be removed before something can make it to the private version. Which one’s a downstream build product of the other doesn’t have an impact on whether we turn feature flags on for the analytics-enabled site before the vanilla version, and IMO either way we go here it’s still going to make the most engineering sense to keep Pendo-specific stuff out of the main “web” repo.

I also don’t think we should change the domain name of the private version, though this is a separable concern from the above. Users with a native wallet created on app.shapeshift.com didn’t expect analytics, and locally-stored data (including native wallets) are tied to domain names, meaning that if we change the URL of the private site we effectively require them to accept analytics to access their wallet. Instead, I propose that when the user accepts on the “please use alpha” splash screen, we seamlessly redirect them to the alpha URL. We’d store their preference so the redirect would occur automatically from then on, and we’d still primarily market and distribute app.shapeshift.com. This would provide a mostly-seamless experience while still respecting the cross-origin security boundary.

The only actual addendum I think we need made to the original proposal is to specify that we’re switching from a default-private experience to a choice-required one, though I personally also think we should explicitly specify that any changes to the scope of the informed consent prompt should also require affirmative user consent.

[0xdef1cafe]

Fully in support of this proposal.

A vote against this proposal is a vote against helping us find product market fit.

[0xApotheosis]

Thanks @BeDiggy, this has my support.

This is a moment where a concession is required to improve our chances of achieving the DAOs long-term mission.

Default opt-in will improve the DAOs ability to create products that users want, and use - and if we execute well: increase our revenue.

[MrNerdHair]

I think we’re all in agreement in principle here, but cementing that agreement in specific language is important. The details are the important part here, and I don’t think the specification language here is ready to go.

• My brain can’t parse “default opt-in” grammatically. Something can either be the default, or it can be opt-in, but it can’t be both. Clarity here is essential.
• This proposal shouldn’t specify implementation details such as CI/CI build order and usage of feature flags.
• The single origin policy is the basis of security on web, and user data – including wallets – is tied to specific domain names. Switching the version that’s deployed to app.shapeshift.com to the version with analytics will affect existing native wallet users by effectively migrating their wallets to the analytics version without their consent, and to the best of my recollection wasn’t something discussed in the recent community call.

My takeaway from the community call’s consensus on this subject is that the actual changes we want from what was approved previously are actually very minimal – specifically, just authorization to make the new-user experience an affirmative choice rather than having Alpha being opt-in. I’m happy to support that change, but I’d like it to be clear that that’s all that’s being contemplated. If there’s more that’s needed, we need to enumerate those needs and discuss them in detail.

[BeDiggy]

On point 3: Any user with a native wallet would immediately see that this version has analytics, when they return. If they choose to continue, they can. If they don’t want to, they’d use the private version. We therefore, would not have any anonymized click data on their native wallet if they choose not to use the analytics version.

On Point 2: Agreed, I will remove/ simplify this post when I move it for polling.

On Point 1: the majority of applications that people use have analytics in their default/only version. This is not a new concept, we are simply informing the user and giving them a choice.

[MrNerdHair]

As I understand the language involved, “default” means what the user gets if they don’t make a choice, while “opt-in” or “opt-out” mean that the user will get one experience by default, and another if they express a preference. If we force the user to choose, there’s no default, and no problem.

This might sound super pedantic, but I feel it’s a critical point. I’m against delivering analytics by default; I’m for explicit user choice, even to the point of a call-to-action on the private site asking them to choose to use the one with analytics.

To my third point above, however, wallets aren’t interchangeable between domains, and that’s not trivial to change from a technical perspective. Therefore, if app.shapeshift.com becomes the version with analytics, an existing user that clicks through to the private version won’t be able to access their existing native wallet(s).

[BeDiggy]

They can import their native wallet into the private version if that’s the one they choose to use.