Non-Functional Questions v2.1.1
ID | S63 |
|---|---|
Version | 2.1.1 |
Type | Overarching Standard |
Status | Retired |
Publication Date | Jul 6, 2022 |
Framework(s) |
Description
Non-functional requirements describe how the Solution works, not what the Solution should do, and serve as constraints or restrictions on the design of the Solution. The following non-functional questions are to enable NHS Digital to assess the risk associated with the Compliance Assessment of the Solution against the overarching Service Management, Information Governance, Security, Testing, and Business Continuity and Disaster Recovery Standards. Supplier’s answers will also demonstrate they understand the non functional landscape of the market they are selling into.
Non-Functional Questions Model
The non-functional questions (NFQs) model illustrated within Figure A attempts to show the relationships between the Supplier Solution and the non-functional requirements (NFRs) required to meet the Overarching Standards, provide appropriate levels of service and, where relevant, satisfy the required service SLAs.
Suppliers are responsible for ensuring the non-functional elements of their Solutions are capable of meeting any defined service levels for specific Capabilities provided, as well as any other NFRs specified by NHS Digital across other Standards.
Suppliers must provide an appropriate answer to the non-functional questions - the level of information and evidence required in the answer will depend on the Capabilities and scale of deployment of the Supplier’s Solution and this will be determined and specified in the guidance provided as part of the Capability Mapping and Standards Compliance On-Boarding process.
Figure A - Non-Functional Questions Model
Non-Functional Categories
Usability
ID | Sub-category | Description | Evidence Example | Assessment Criteria |
|---|---|---|---|---|
GP-NFQ-U-3 | Supported browser versions | Which browsers are supported and what are the minimum and recommended versions? How do you approach the deprecation and uplift of browser versions? | Provide declaration and any supporting documented evidence and rationale. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-U-6 | UI standards | What user interface standards does the Solution meet? Are you following ISO 9241-210:2010 - the ‘six principles for human centred design'? | Provide declaration and any supporting documented evidence and rationale. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-U-7 | Supported devices | What devices are supported by the Solution that can be used and accessed by end users? | Provide type of devices supported, compatible with the Solution and any supporting documented evidence and rationale. | Demonstrate that the Solution is securely accessed and used by end users on supported devices through authentication and authorisation mechanisms in line with NHS Digital policies. |
GP-NFQ-U-8 | Accessibility | Describe how the Solution is accessed by / available to the end user? | Provide any supporting documents, and mechanism to access the Solution, for example, how to access the Solution in the event of interruption and how availability of the event is managed by the Solution | Demonstrate that the Solution is securely accessed by the end user via the provided mechanism. Demonstrate the Solution is accessed and available to the end user via provided the mechanism if interrupted. Demonstrate that event will be available for appropriate length of time. |
Performance and Scalability
ID | Sub-category | Description | Evidence Example | Assessment Criteria |
|---|---|---|---|---|
GP-NFQ-PS-3 | Number of concurrent sessions | What is the peak number of concurrent user sessions the Solution can support? | Provide a value, show how the figure is derived, including any considerations which may impact this value. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-PS-5 | Response times | What are the expected response times in a given percentage of cases for a given operation e.g. Login or Open Patient Record? Provide different types of operation including command and query operations. | Provide values, show how the figures are derived, including any considerations which may impact these values. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-PS-6 | Transactions per second | How many user transactions per second are supported? Define what a user transaction consists of and how it is measured. | Provide a value, show how the figure is derived, including any considerations which may impact this value. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-PS-7 | Online service transaction volumes | How many online batch transactions per second are supported? Define what a batch transaction consists of and how it is measured. | Provide a value, show how the figure is derived, including any considerations which may impact this value. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-PS-11 | System Response Times | Describe how measurements are calculated for system response times to meet SLAs. | Provide values, show how the figures are derived, including any considerations which may impact the values. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-PS-12 | End User Interaction Timings | Describe how measurements are calculated for end user interaction timings to meet SLAs. | Provide values, show how the figures are derived, including any considerations which may impact the values. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-PS-13 | Interface Mechanism Response Times | Describe how measurements are calculated for interface mechanism response times to meet SLAs. | Provide values, show how the figures are derived, including any considerations which may impact the values. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-PS-14 | Video and audio and picture / image quality | What are expected minimum requirements for good quality of video and audio and pictures / images sent via the system? | Provide minimum specification to get better quality of video and audio and pictures / images. | Demonstrate the following minimum requirements for better video / image, audio quality: Minimum resolution of up to 720p or higher is required for HD ready. End-to-end latency must be lower than 150 ms to avoid poor performance of image Audio encoded at a minimum of 16 Kbit/s |
Volume and Performance
ID | Sub-category | Description | Evidence Example | Assessment Criteria |
|---|---|---|---|---|
GP-NFQ-VP-1 | Model | Describe what volumetric model(s) are provided to enable volume and performance tests encompassing load, ramp, stress and soak phases? | Provide model(s), show how the model(s) are derived, including any considerations which may impact the model(s). | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-VP-2 | V&P Performance | What V&P work has been undertaken to ensure the Solution will meet expected volume & performance SLAs. | Provide statement of V&P assessment activities and associated test reports. | Demonstrate systematic approach of how the evidence is derived. |
GP-NFQ-VP-3 | Spine | What V & P testing of National Spine interactions (e.g. Personal Demographics Service, Summary Care Record, e-Referral Service) is planned, to ensure Supplier’s Solution has no impact on National Spine services? | Provide statement of testing and test coverage | Validate test coverage. |
Non Functional Testing
ID | Sub-category | Description | Evidence Example | Assessment Criteria |
|---|---|---|---|---|
GP-NFQ-T-1 | Test Plan | Describe what test plan and schedule of non-functional testing is in place and with what associated documentation? | Provide test plan to include sufficient details of scope and coverage. | Validate scope and non-functional test coverage. |
GP-NFQ-T-2 | Data Protection Testing | What Data Protection testing is in place to show legislation is met? | Provide statement of test activity, the scope, coverage and attributable legislation | Validate scope and non-functional test coverage. |
GP-NFQ-T-3 | ITHC / Penetration Testing | What ITHC / penetration testing is planned? Does the penetration test provider have one of these qualifications: Check, CREST or TIGER? | Provide statement of test activity, the scope, testing results and coverage | Validate scope, process and testing, testing results and any recommendations requiring corrective actions completed and acted upon. |
GP-NFQ-T-4 | Ready for Operations (RFO) Testing | What RFO testing is planned? | Provide details of RFO test coverage, examples: • Application / component failure and recovery • Monitoring and reporting of SLAs | Validate test coverage. |
Recoverability
ID | Evidence Example | Assessment Criteria | ||
|---|---|---|---|---|
GP-NFQ-R-1 | Disaster recovery point objective (RPO) | What RPO is possible with this Solution and how does that map to the required SLA? | Provide a value, show how the figure is derived, including any considerations which may impact this value. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-R-2 | Disaster recovery time objective (RTO) | What RTO is possible with this Solution and how does that map to the required SLA? | Provide a value, show how the figure is derived, including any considerations which may impact this value. | Demonstrate systematic approach and rationale of how evidence is derived. |
Backup capability
ID | Sub-category | Description | Evidence Example | Assessment Criteria |
|---|---|---|---|---|
GP-NFQ-B-1 | Capability | Describe in detail what backup and recovery mechanisms are implemented. | Confirm formal processes and mechanisms are in place. Provide design and documentation of implementation and operational processes. | Demonstrate systematic approach and rationale of design, implementation and processes. |
GP-NFQ-B-2 | Validation and Testing | Describe what testing has been undertaken to demonstrate the mechanism and processes support your operational requirements. | Provide appropriate statement of test activity, coverage and outcomes | Demonstrate systematic approach and rationale of design, implementation and processes. |
GP-NFQ-B-3 | Periodic Testing | Describe what periodic testing of backup and recovery is undertaken? | Provide appropriate statement of test activity, coverage and outcomes. | Demonstrate systematic approach and rationale of design, implementation and processes. |
Availability
ID | Sub-category | Description | Evidence Example | Assessment Criteria |
|---|---|---|---|---|
GP-NFQ-A-1 | Availability | What is your service availability during agreed service hours. 90% ("one nine") 36.5 days 72 hours 16.8 hours | Provide a value, show how the figure is derived, including any considerations which may impact this value. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-A-2 | Service hours | What are the standard business operating hours the Solution needs to be available for users? | Provide details of service hours, where a solution may deliver a number of capabilities, a declaration of appropriate service hours is required. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-A-3 | Scheduled maintenance windows | What are the requirements for any scheduled periods of unavailability to perform Solution maintenance? | Provide statement, the underlying rationale and any impact considerations. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-A-4 | External system impacts | Does the availability of any external systems affect this Solution? | Provide statement, the underlying rationale and any impact considerations. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-A-5 | Fault tolerance | Describe any Single Points of Failure (SPOF) within the Solution or external that the Solution relies upon for operation. | Provide statement of any SPOF inherent within the Solution and where applicable, any associated mitigation.. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-A-6 | Degradability | Describe the ability of the solution to operate with reduced capacity or functionality in the event of an unexpected event, e.g. site failure | Provide details of any reduced operations capability, the levels of functionality and details of supporting documentation | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-A-7 | Disaster recovery minimum recovery operating level (MROL) | In the event of a disaster what are the minimum business services that need to be recovered to continue operating? | Provide details of any reduced operations capability, the levels of functionality and details of supporting documentation | Demonstrate systematic approach and rationale of how evidence is derived. |
Resilience
ID | Sub-category | Description | Evidence Example | Assessment Criteria |
|---|---|---|---|---|
GP-NFQ-RS-1 | Service Level Agreements (SLAs) | Describe how the hardware and software design supports the SLAs? | Provide details of how solution design supports the delivery of any SLA declaration, including any impact considerations. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-RS-2 | Hardware Design | Describe what level of resilience is provided in the hardware design? | Provide documented evidence and rationale, including any impact considerations. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-RS-3 | Software Design | Describe what level of resilience is provided in the software design? | Provide documented evidence and rationale, including any impact considerations. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-RS-4 | Business Impact of Service Incidents | Describe what mapping of business impacts of service incidents against levels of resilience has been undertaken? | Provide documented evidence and rationale, including any impact considerations. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-RS-5 | Component Failure Impact Analysis | Describe what CIFA documentation has been produced? | Confirm CFIA analysis has been completed, provide documented evidence of analysis. | Demonstrate systematic approach and rationale of how evidence is derived. |
GP-NFQ-RS-6 | Network quality / performance | Describe what hardware / software requirements are supported for good quality of video, minimum resources required for good quality of video? | Provide minimum specification to get better quality of video, details of resources like data usage, costs associated to claim economic benefits of your Solution. | Demonstrate the following minimum requirements for better network quality: minimum call speed/bandwidth 500 Kbps Upload and download speed: minimum 5 Mbps download and 2 Mbps upload speed Confirm that data usage and cost associated is at minimum level. |
Information Governance
ID | Sub-category | Description | Evidence Example | Assessment Criteria |
|---|---|---|---|---|
GP-NFQ-IG-1 | Data retention | What NHS Digital data retention policies are enforced by the Solution? See GP-IG-14.2-4 in the Information Governance Standard. | Provide documented evidence and rationale, including any impact considerations. or Provide list of policies | Demonstrate systematic approach and rationale of how evidence is derived. or How well do the policies map to requirements and legislation |
GP-NFQ-IG-2 |