Project Confusion in Transition Management

This article is a compilation of notes on acceptance testing in my region.

Acceptance tests are conducted when a system or deliverable is defined as ready for acceptance. How a deliverable is determined as ready or complete is a matter for agreement between the project owner i.e. customer and the project team aka supplier. Generally the completeness of a deliverable should cover all aspects of contracted requirements. People's expectations in particular and specified granularity on requirement details for quality matters, can influence and change what will ultimately constitute the completeness of a project deliverable.

Acceptance tests are conducted in the presence of the customer, typically after the customer has agreed to take over a deliverable.

Testing considers the location and are conducted at both the following (always for large projects):

• Staging site which may be at the Factory where a system or product is created, or at a location where the system or product is setup prior to delivery to the customer's site.
• Actual site where the project is deployed

In brief, the following are performed leading to the actual test (see test categories):

1. Agreeing on what will constitute acceptance testing for that test category - especially if not already specified or detailed.
2. Document test - review test procedures and walkthrough the steps, a dry run to test a deliverable, system or product.
3. Preliminary test - a real test conducted with or without the presence of the customer to validate the tests and readiness for acceptance testing. This will be the last save-face stage or opportunity where flaws are affirmed and finally rectified.
4. Conduct actual or final test e.g. Site Acceptance Tests

The different test categories in sequence:

1. Factory Acceptance Tests - this exclude testing at subcontractors', suppliers' as they are not known to your customer
2. Site Acceptance Tests - that whatever delivered from Factory are as-is. Functional tests may be conducted separately. 
3. Site Security Acceptance Tests or Systems Security Acceptance Tests - may involve separate tests such as physical security, checks on personnel, and cyber security.
4. Reliability Acceptance Tests - this exclude the warranty period
5. Audits by independent internal and-or external parties - this exclude any visit by the authorities

The number of tests can easily double when we factor in requirement for preliminary testing (the exception being the reliability test itself and audits - we can't tell auditors what to do). And yes, there will be corrections in every test category.

Upon completion of each test from the category list above - at the least, will incur the following activities:

1. Review test results
2. Ratify test results
3. Massage the customer to Accept test results

Note that acceptance tests may be conducted:

• Before handling over and taking over (HOTO) the deliverable
• Throughout the HOTO process 
• After HOTO of a deliverable!
• Or in any manner as agreed between the customer and supplier.

A best practice approach to acceptance testing:

• Let the domain experts determine and decide on what to test for acceptance of deliverables.
• Corroborate testing with what was contracted.
• Seek the agreement and approval of all stakeholders especially the customer on the tests. The exception being - perhaps if you are the empowered authority or auditor who can barge in to test for compliance.

Read also User Acceptance Tests (documentation issues)

* I'll keep this as a live document. Hmm ..

This brief is intended to document & serve as a general guide on what typical technology features 'to look out for' when confronted with new ICT application projects or opportunities. At a high level overview of common system functions which are necessary to implement useful tech products, gleaned from experience. These notes may be biased towards using the Cloud.

IMO a useful piece of software should be designed & developed with minimally the following broad features:

1. Parameters. Provide for editable system /application /administrative data instead of hardcoding values; e.g. background color may be softcoded as a parameter instead of hardcoded to a specific color code.
2. Logs. Wherever possible log a record for each & every task /action /response performed. 
3. Fault reporting. Application broadcasts & accepts query as to its operating status; e.g. using SNMP on separate channels with remote access capability for maintenance & troubleshooting.
4. Interface to common incl. popular communication medium i.e. email, large file transfer, text messaging & SMS. Additionally the application may need to interface with IoT sensors & smart devices e.g. handphones. Interface incl. to virtual devices where applicable.
5. High availability e.g. use the Cloud to maintain up-time, transfer downtime risks to external provider, & manage quick releases/rollback. 
6. SSO with options to let users specify & access using their own devices.
7. Users maintain & update their own profiles in compliance with local laws, use policies, & user agreements.
8. Open APIs (somewhat) that would enable add-on applications from 3rd party developers & at later dates.
9. A permanent history of hotfixes, patches & updates since first-use, accessible to users. This may include records of e.g. formal backup of data files and restoration from backup if any.

The list above surmises my preferences. Yours may differ. And there are many other features, some which are essential to consider & in detail when specifying functional requirements of a piece of application software /device /in combination. Now that the application has some form of character, who uses it should be considered next.

The application should distinguish without elaborating, the following roles:

• Administrators
• Internal users
• External users

Users have their expectations too. A good piece of application should be accessible anywhere & anytime. There are limits though if the application is dependent on something or things beyond your control (design /development /operational).

High availability & ease of use to a user would be expectations for e.g. in software:

• save as draft feature i.e. saving automatically a form or a piece of work & recovering from a fault with most if not all previous inputs/data intact
• import bulk data to ease data entry
• input/translate different but common file formats e.g. images, audio etc.
• output to common file formats e.g. pdf, Excel based worksheets
• friendly user interface e.g. drag & drop files, auto login /re-login with facial recognition, thumb or finger prints.

Are you ready to accede to requirements? Assuming money matters are outside your purview e.g. procurement of licenses, additional considerations would include ensuring:

1. Conformity to coding standards that will meet
2. software testing requirements;
3. adherence to software & communications security protocols & standards,
4. personal data protection,
5. maintainability ↓,
6. a consistent framework for usability & review for revision or upgrade, of the application or system.

Few pointers to think about for maintenance & maintainability of the application include:

• Whether maintained internally i.e. by the company, or
• Outsource to external vendor
• Ease of troubleshooting workflows /data processing to resolve disputes ~ use/maintenance/operational.
• Documentation; ease of access to & clarity of information; is documentation up-to-date or simply lodged in someone's head.