If you have done any modern web development as an engineer or manager you would be familiar with the structure of development, staging and production environments. For those not quite familiar, briefly they are built environments (i.e. fully functioning product) that are used in different parts of your code development cycle.
Development environment is the bundle of code that most typically runs on the computers of the engineers. They have the ability to function without needing actual resources other than the computer they use. This is because the developers need a good replica of the environment with the exact technology but without any possibility to break anything. So they are not connected to the real world of their app.
Staging environments are more close to what the final product environment would be. If the app is on the cloud the stage is on the cloud. It uses connections and databases just like the actual app but it does not in fact have any real user data. Developed features get pushed here to try them out without messing with existing user data.
When everything is ready to go you merge the code into the production environment where in most cases it goes live and the code has the potential to change actual user data.
This structure is useful because code can be disruptive. Even if you could easily roll back changes the users want an uninterrupted environment without surprises. This also allows engineers to experiment with their code. When all their work is local to their computers and they use versioning systems, they can change different aspects of the app without any problems.
Do product teams needs a stage environment to test out product ideas or is that even possible?
We have often used feature flags in production environments and allowed specific clients to see certain changes. This works well but it has limitations. Because we push code to production it can’t be a major alteration. We have mainly used it for product ideas that we know are going to production or we have had separate pages that are not intertwined with any other code, so we had complete freedom. Even if that’s the case however, changes that would be made to the database would still affect other users as well and database migrations are some of the more complex issues we can run into in production.
So it would make sense if we could have a stage environment for testing product ideas but it would miss a key component: users. Without actual user data and actual user events we wouldn’t be getting all the information we need. However it might also make sense to get paid users or hypothetical users just for stage environments. Especially for B2C products this would be much easier to generate. Another option would be to replicate production databases and keep them separate so that any testing we do would reflect changes in real data but not touch actual production data.
This kind of stage environment can have multiple benefits: It would copy the production environment much better than any mockups we can provide It would allow us to test our elements with real data. For instance if we have paginated a view we can see generally how many pages that would result in. If we dynamically list items in our new feature for users we can see how many items get listed and how this affects the layout. We can see if existing dynamic text from the users fit the layout or if we have taken care of dynamic text spacing properly. We can see if existing settings and preferences for users affect their view of our feature. We can invite testers and give them accounts that they can explore themselves. We can observe some generated assets like charts, dashboards to see if they will be meaningful for clients. We can run extra scripts that would normally slow down the production pages without worrying about production level performance. We don’t need to worry about outages and users losing access to important features due not to code but the product decision “bugs” that we made.
This would also be extra effort and cost some money to run, plus it would introduce another environment for code to sync with. But I think it would be worthwhile in the long run to give product teams much more flexibility in trying things out and removes our over reliance on mockups and prototypes.