WYROM’S WORDS: PM Wyrom spends some time discussing QC and Beta Testing, There’s a LOT to read, and it is important enough that I’m not going to try to summarize it:
Text of Discord Post follows:
PM Wyrom
QC and Beta Testing
on 8 March 2023 8:48 PM
from the Official GemStoneIV Discord Forums
in the General CategoryHey everyone, I’d like to address two topics that have come up a lot recently, which are quality control and beta testing.
Quality Control (QC)
In GemStone, we use the term QC as the final step a project takes between the GameMaster to production (the live games). The QC process isn’t as robust as other game studios or companies use the term. The QC process is typically the inspection and/or deliverable(s) peer review. Our quality assurance process is typically called “approvals.” This is done by a few staff members. Individual GameMasters are managed by the Assistant Product Managers and they perform the individual approvals. We also have project leads who perform some of the approvals. Finally we have team approvals which is where a lot of the player-facing approvals tend to happen (Dev approves functional script, combat, magic, and technical systems, Events approve specific storyline and pay event needs, Production approves subscription based projects, and Player experience approves a lot of the customer service, training, and tutorial systems).
There are two forms of QC, technical and creative. Shops all go through creative. Item scripts go through both. Systems are usually both, but not all the time. The testing process is done by the GameMaster(s) working the project. For example, if I’m coding a new script, I am responsible for fully testing this script. Which means using it like a player would use it. Some QCers will also test the scripts, but it isn’t a requirement to do this, since the QC process for a script is going through the actual code line by line and making sure we’re using the proper routines, that the code isn’t going to create unwanted actions, null returns, etc. When I first started, an item script typically was around 300 to 500 lines of code. Lately, I see item scripts getting as large as 2,000 lines of code. It’s grown quite a bit as we’ve allowed GameMasters to do a lot more in their scripts over the years.
Beta Testing
GemStone uses the Interactive Fiction Engine (IFE), which is software created to run the games. We sometimes refer to this as the C-level engine. The engine controls a lot of the core functions of the game, such as randomization, roundtime, how specific exist profiles work, and a number of things that would be tough to go into without teaching a class. Anything that is done to the engine requires the game to be restarted, think of it like a patch in any other game. The GemStone Language (GSL) is a programming language used to create everything in the game that you interact with, such as new features, verbs, creatures, items, etc. The engine is not updated often, but we do maintenance on it from time to time. This is done by our software engineer at Simutronics. GSL is where myself and the GameMasters all work though.
GSL allows us to push updates out to the game on the fly, we don’t have to create a downloadable patch as explained earlier, a lot of other games might refer to this as a hotfix. For us, it’s normal day. But this allows us to put updates out that aren’t necessarily in their final state. Updates are generally flexible and we typically involve feedback after something is released to make it better. This is sometimes referred as a quality of life update, but in other cases it just resolves issues that didn’t come up during the initial project. This is actually a pretty appealing to other developers with how GSL and the IFE work, and I’ve even had the chance to speak about it with other studios within the Stillfront Group. But it comes with the fact that a lot of systems launch in beta. Because individual updates aren’t ever in a final state and we can perform unlimited updates at any time, nothing is tied directly to the game itself, but the updates. That’s put GemStone (and DragonRealms) in a perpetual beta state. This is by design so that we can always be flexible. It doesn’t mean we try to push out buggy updates, but it is a byproduct of it. Especially when different systems interact with each other (a good example is RotFlares and BriarFlares. If BriarFlares receives an update, it can have buggy results on RotFlares. No part of any of our processes will ever test RotFlares on new updates, the RotFlares testing would need to be done with by the script owner (me), but I might not know what changed in BriarFlares. It’s why no sane person does subscripting).
Final Words
I’m hoping to clarify what the QA, QC, and testing process is like in GemStone. I’m also letting you know, debut releases will always be in a beta state. It’s the nature of our game. If you don’t want to jump into that or you find that you are really only interested in a finalized release, I would highly recommend standing clear of debut releases until you can read up on the state of the release. We have cleaned up a lot of the QC process going forward, but I’m just covering the fact that a lot can change during the development and release that might not also be known to the coder.
more info down the thread:
Q: Hi Wyrom, what is the process for moving a script from beta state to finalized state?Wyrom
A: There really isn’t a process for that. In fact, even on play.net it stated that the game remains in beta for most of its features. I think it was removed at some point, but it was on there for awhile near the whole letter to the parents thing (another outdated piece of the game and site, we’re not rated E anymore).
The coding GM has all the same tools that a player does, so it’s on the individual GMs to test whatever the script does. It’s not so much on the QCer.
As for tossing on Test, it will be up to the individual GMs. The problem with testing is then there will be an endless number of suggestions.
Mentors and GameHosts aren’t under NDA. Most time testers aren’t always the best target audience. I did send out a note to our staff to make sure they are testing their scripts better as it’s becoming an issue where the QCer is taking the flak for things. Most the bugs lately aren’t even script errors, just not getting the results you should be getting.
If we were to do testing, I would likely pull a group aside that would be dedicated to it.
He says there will be more after a meeting, so use the link below to follow the discussion on Discord as it continues.
Recent TownCrier newsletters.
