Package/Software Testing
This is one of those questions that you get from the "higher ups" that you scratch your head and wonder, "what would be the point in attempting to do this?" But then, you go ahaead and do it anyways because your being paid to.
I've been asked to see if there is a way or if there are tools to improve our software packages.
What we do now is basically:
If you have any ideas as to how to improve apon this, I'd like your input.
I've been asked to see if there is a way or if there are tools to improve our software packages.
What we do now is basically:
- Install the software manually to see what it does.
- Package the software and/or make customizations as well as automate the install.
- Install the new package and make sure it works.
- Have the client test the software and make sure it works to their satisfaction.
If you have any ideas as to how to improve apon this, I'd like your input.
0 Comments
[ + ] Show comments
Answers (11)
Please log in to answer
Posted by:
dunnpy
13 years ago
The only things I can add are:
'Package Documentation' - how did you create the package, any 'gotchas' to be aware of for the next release, custom actions or deviation to your defined standards.
'QA Documentation' - peer review of your package output to check that it's fit for purpose and conforms to your standards.
'Package Documentation' - how did you create the package, any 'gotchas' to be aware of for the next release, custom actions or deviation to your defined standards.
'QA Documentation' - peer review of your package output to check that it's fit for purpose and conforms to your standards.
Posted by:
jmaclaurin
13 years ago
We have automated the basics and prep for our package creation which includes document templates that are prepopulated, pre-written cmd install scripts, MSI templates, reg keys, etc, all of which is done in a vb script, but I consider that as part of the packaging steps. As for additional documentation, every package has a packaging notes doc that we include the steps used for creating the package and anything unusual. We also have an Internal Verification (IV) process (peer testing) which is done pre-client testing. So really, every package gets tested at least 3 times, packager, IV and then client, before it goes out for public use.
There are a few theories floating around with regards to the automation that we include in our packaging processes. The biggest concern with over automation is the packagers aren't learning anything so they don't know how to troubleshoot. The other side is that there is less chance for mistakes to be made. I personally think we have a pretty good balance and wouldn't want it to be changed much more. I guess management looks at it differently and wants everything automated including the testing steps.
There are a few theories floating around with regards to the automation that we include in our packaging processes. The biggest concern with over automation is the packagers aren't learning anything so they don't know how to troubleshoot. The other side is that there is less chance for mistakes to be made. I personally think we have a pretty good balance and wouldn't want it to be changed much more. I guess management looks at it differently and wants everything automated including the testing steps.
Posted by:
Arminius
13 years ago
I don't think that there are more steps you could add at this level without creating extra overhead in the process. There are tweaks to add, however, that may make your management happier.
1. document (see Dunnpy above). Ensure that you have correct installation methods from SME/owner.
2. duh, this is what we do. Perhaps adding the level of user interaction we want the customer to have with installs, such as "99% of installs will be fully automated with no end-user interaction". Bossess love metrics
3. Define make sure it works - as packagers, we aren't experts in the software. Does it launch without errors, configuration needs, or other messages? That may be a good thing to document as a standard
4. Are app owners accountable for this step of the process? At one job I had, if there wasn't an app owner, then no one could approve a software release and it didn't go out. Period, end of story. Made it nice for me that someone else had to say it was configured correctly.
This brings up the question: what issues are there with your software packages? You mention " if there is a way or if there are tools to improve our software packages". I'd look there and see what issues there are and address them - if you go on a wholesale improvement program and don't address management's issue, you've spent a lot of time delivering something that wasn't what was asked for.
Good luck...
1. document (see Dunnpy above). Ensure that you have correct installation methods from SME/owner.
2. duh, this is what we do. Perhaps adding the level of user interaction we want the customer to have with installs, such as "99% of installs will be fully automated with no end-user interaction". Bossess love metrics
3. Define make sure it works - as packagers, we aren't experts in the software. Does it launch without errors, configuration needs, or other messages? That may be a good thing to document as a standard
4. Are app owners accountable for this step of the process? At one job I had, if there wasn't an app owner, then no one could approve a software release and it didn't go out. Period, end of story. Made it nice for me that someone else had to say it was configured correctly.
This brings up the question: what issues are there with your software packages? You mention " if there is a way or if there are tools to improve our software packages". I'd look there and see what issues there are and address them - if you go on a wholesale improvement program and don't address management's issue, you've spent a lot of time delivering something that wasn't what was asked for.
Good luck...
Posted by:
jmaclaurin
13 years ago
Like you, 99% or our packages are automated providing it supports it and the effort to do so is worth it. The problems we usually end up with are caused by a vendor that supplied a crap package to begin with.
Yes, we just launch the app to be sure it works and then let the client do the actual test because they are the owner and product experts.
Yes, we just launch the app to be sure it works and then let the client do the actual test because they are the owner and product experts.
Posted by:
jmcfadyen
13 years ago
personally I like to automate everything.
you mentioned the packages don't learn if that is the case, (I found that too) so in my automation scripts I put in training techniques which can be turned on or off easily.
you mentioned you use a vb script (is this to write registry keys?) I am confused with the comment if that is what you are doing you would be better off automating the addition of those registry keys into the packages as registry keys in the registry table. .
not sure if i misunderstood what you were saying there.
you mentioned the packages don't learn if that is the case, (I found that too) so in my automation scripts I put in training techniques which can be turned on or off easily.
you mentioned you use a vb script (is this to write registry keys?) I am confused with the comment if that is what you are doing you would be better off automating the addition of those registry keys into the packages as registry keys in the registry table. .
not sure if i misunderstood what you were saying there.
Posted by:
mazessj
13 years ago
jmaclaurin, I'd be interested to hear more about your automations. Would you be able to post script examples, or are there intellectual property concerns?
jmcfadyen, I'd be interested in seeing what you've done, too (again if allowed).
I feel I have things working pretty well where I work, but there's always room for improvement.
Thanks.
jmcfadyen, I'd be interested in seeing what you've done, too (again if allowed).
I feel I have things working pretty well where I work, but there's always room for improvement.
Thanks.
Posted by:
jmaclaurin
13 years ago
I have generated scripts for automation in so many different environments, it would probably be lost in translation if I were to post them here. And yes, in the current environment, intellectual property comes into play because it is soooo costum for their very specific needs.
I can, however provide an overview of the steps involved.
VBScript is launched:
Prior to this, AS/IS templates are created and prepopulated with the standard settings and contact info for use in MSI creation.
There is a lot more than this, but you can see how using available technologies and easily take a lot of the monotony and errors out of common steps.
I can, however provide an overview of the steps involved.
VBScript is launched:
- a dialogue is provided requesting the particulars (Name, Version, Size, etc.) for the package to be automated.
- folder structure is created to store the package on the development server
- document templates are copied into the folder structure and fields within the documents are autopopulated with the previous information
- appropriate preconfigured CMD file is selected and copied into the folder structure.
- tag files and reg entries are created to be included with the package when deployed for inventorying purposes and are copied into the folder structure
Prior to this, AS/IS templates are created and prepopulated with the standard settings and contact info for use in MSI creation.
There is a lot more than this, but you can see how using available technologies and easily take a lot of the monotony and errors out of common steps.
Posted by:
mazessj
13 years ago
An overview was really what I was looking for to start anyway, so thanks for posting. It looks like we're already doing a lot of the things on your list, only our process is currently driven by Process Templates in Wise Package Studio. And since WPS is going the way of the dodo bird, I may eventually have to redesign it.
1. Some of this information is covered by the initial project setup (Package Name, Package Version, Application Name, etc.)
2, 3. Much is scripted via vbscript.
3. Autopopulation in the Integration Notes document is handled to a degree. It might be nice if I could prepopulate the PACKAGENAME property in my SetupCaptures; I think I know how to do it, I just haven't taken the time. Probably more difficult to do for new transforms.
4. I suppose I should consider creating a preconfigured CMD file like you, configured to simulate a deployment as closely as possible -- probably by using psexec to run under the SYSTEM account. We do autopopulate a control file, though, for a custom Setup front-end that we use for every package. (This custom Setup handles uninstalling previous versions, killing tasks, pre-requisites, post-requisites, etc.)
5. Your tag files & reg entries I'm guessing are equivalent to our "footprint" -- we write a reg and log entry at the beginning and the end of the installation that date-stamps the installation and its success or failure. The footprint code is in our Windows Application project template. PACKAGENAME is set to 0 by default and has to be filled in by the packager. Currently, the installation fails if the PACKAGENAME field isn't filled in, so while it's not currently autopopulated, at least a package won't work if this property isn't set.
I agree that automating as much of this stuff (project setup, data entry, etc.) as possible is definitely a good thing. It allows the packager to focus on packaging.
1. Some of this information is covered by the initial project setup (Package Name, Package Version, Application Name, etc.)
2, 3. Much is scripted via vbscript.
3. Autopopulation in the Integration Notes document is handled to a degree. It might be nice if I could prepopulate the PACKAGENAME property in my SetupCaptures; I think I know how to do it, I just haven't taken the time. Probably more difficult to do for new transforms.
4. I suppose I should consider creating a preconfigured CMD file like you, configured to simulate a deployment as closely as possible -- probably by using psexec to run under the SYSTEM account. We do autopopulate a control file, though, for a custom Setup front-end that we use for every package. (This custom Setup handles uninstalling previous versions, killing tasks, pre-requisites, post-requisites, etc.)
5. Your tag files & reg entries I'm guessing are equivalent to our "footprint" -- we write a reg and log entry at the beginning and the end of the installation that date-stamps the installation and its success or failure. The footprint code is in our Windows Application project template. PACKAGENAME is set to 0 by default and has to be filled in by the packager. Currently, the installation fails if the PACKAGENAME field isn't filled in, so while it's not currently autopopulated, at least a package won't work if this property isn't set.
I agree that automating as much of this stuff (project setup, data entry, etc.) as possible is definitely a good thing. It allows the packager to focus on packaging.
Posted by:
jmaclaurin
13 years ago
I'd say it was definately worth it, and especially so when we had to repackage some 170+/- packages for windows 7. We brought in people from other departments that had limited to no packaging experience so having most of the processes automated and preconfigured sure made the chance for error less likely. And yes, I'd agree that it does take out the wasteful time of performing the tedious tasks. I'd say we save about 1-2 hours and removes a lot of the conflict/stress if it were done wrong.
Posted by:
MPA2012
13 years ago
I have for that past 15 years have been consulting for large clients that never had stream-lined packaging processes. So I started use Wise script and VBScript to create automated tasks for the packaging template processes in Wise Studio. I since then have created a C# utility that will integrate into Wise or Admin Studio Suites.
I made the utility easy to change as I go to each client and after implementing this into their current packaging environment, I give them the code to continue enhancing the process after I have left.
There is a lot of utilities out there to help you stream-line, but they always never fit the users environment exactly as they want it too. So why spend all that money on the software and resource time, when you can just spend it on resource for someone like me to come in and modify my utility to stream-line your current process more than you have now.
My Utility works in both Studio's, as I mentioned above. I review the clients current process and come up with process flow template and then manually run the tasks to document the steps (that will be displayed to the right of task in Studio) and after completed that i will create or tweak the task to stream-line(automate) the task.
Here are some images of the process template from AdminStudio with explanations.
Notice the first line is the image is the ProjectName. This is the standard name I use to make all the project consistent and easier to stream-line.
VendorName_AppName_AppVer_BuildVer (BuildVer - T01=ThinApp, N01=Native(MSI), A01=AppV, X01=XenApp)
Note: B01, is from an older process and this client wanted the 'B' because they where creating MSI/AppV packages with in one process.
This image shows the folder structure and how my utility keeps every thing in its rightful place.
This image is giving you an example of how I can stream-line to external processes with not leaving the packaging tools.
Most clients use some sort of virtual environment to capture and UAT their re-packaged applications. So I have used VMware API's to control the VMs for the re-packaging efforts.
By using the image above, I started the capture VM with the Repack snapshot and launched a pre-scan/vendor Install/post-scan to end up with the results in the second image above.
So you can see that I work on taking the monotony and user error out of the process and leave the packagers to what they do best, re-package the applications.
P.S. this is a high level over view of what I do for client processes, but it will do anything you need it to stream-line you process.
Hope this helps, and if you have any other questions, feel free to contact me.
-Mike
I made the utility easy to change as I go to each client and after implementing this into their current packaging environment, I give them the code to continue enhancing the process after I have left.
There is a lot of utilities out there to help you stream-line, but they always never fit the users environment exactly as they want it too. So why spend all that money on the software and resource time, when you can just spend it on resource for someone like me to come in and modify my utility to stream-line your current process more than you have now.
My Utility works in both Studio's, as I mentioned above. I review the clients current process and come up with process flow template and then manually run the tasks to document the steps (that will be displayed to the right of task in Studio) and after completed that i will create or tweak the task to stream-line(automate) the task.
Here are some images of the process template from AdminStudio with explanations.
Notice the first line is the image is the ProjectName. This is the standard name I use to make all the project consistent and easier to stream-line.
VendorName_AppName_AppVer_BuildVer (BuildVer - T01=ThinApp, N01=Native(MSI), A01=AppV, X01=XenApp)
Note: B01, is from an older process and this client wanted the 'B' because they where creating MSI/AppV packages with in one process.
This image shows the folder structure and how my utility keeps every thing in its rightful place.
This image is giving you an example of how I can stream-line to external processes with not leaving the packaging tools.
Most clients use some sort of virtual environment to capture and UAT their re-packaged applications. So I have used VMware API's to control the VMs for the re-packaging efforts.
By using the image above, I started the capture VM with the Repack snapshot and launched a pre-scan/vendor Install/post-scan to end up with the results in the second image above.
So you can see that I work on taking the monotony and user error out of the process and leave the packagers to what they do best, re-package the applications.
P.S. this is a high level over view of what I do for client processes, but it will do anything you need it to stream-line you process.
Hope this helps, and if you have any other questions, feel free to contact me.
-Mike
Rating comments in this legacy AppDeploy message board thread won't reorder them,
so that the conversation will remain readable.
so that the conversation will remain readable.