Secure Software Development and Product Security

2020.1

FlowTrack development team follows the latest security best practices when developing software, and automates security testing throughout development lifecycle whenever possible.

Security is integrated into all phases of FlowTrack product development lifecycle, including:

  • Secure Design:

    • App Risk classification
    • Security req definition
    • Secure application design / RFC
    • Threat modeling
    • App data flow analysis
  • Secure Development and Testing:

    • Secure coding guidelines
    • Peer review
    • Security testing (Static secure code analysis)
    • Responsible vulnerability disclosure
  • Remediation:

    • Follows defined vulnerability management lifecycle
    • Ensures no high risk security vulnerability is in production

Details about the FlowTrack software application architecture and security are documented on the product development / engineering wiki.

Policy Statements

FlowTrack policy requires that:

(a) FlowTrack software engineering and product development is required to follow security best practices. Product should be "Secure by Design" and "Secure by Default".

(b) Quality assurance activities must be performed. This may include

  • peer code reviews prior to merging new code into the main development branch (e.g. master branch); and
  • thorough product testing before releasing to production (e.g. unit testing and integration testing).

(c) Risk assessment activities (i.e. threat modeling) must be performed for a new product or major changes to an existing product.

(d) Security requirements must be defined, tracked, and implemented.

(e) Security analysis must be performed for any open source software and/or third-party components and dependencies included in FlowTrack software products.

(f) Static application security testing (SAST) must be performed throughout development and prior to each release.

(g) Dynamic application security testing (DAST) must be performed for major application releases or as determined by the Security Officer

(h) All critical or high severity security findings must be remediated prior to each release.

(i) All critical or high severity vulnerabilities discovered post release must be remediated in the next release or within 30 days, whichever is sooner.

(j) Any exception to the remediation of a finding must be documented and approved by the security team.

Controls and Procedures

Software Development Process

Overview & Release Strategy

Software development at FlowTrack follows a release strategy that provides traceability for production software changes. Features, enhancements, and bugs are written up as Issues in Jira. An engineer on a small team proposes changes necessary and creates a review for the team (AWS CodeCommit). Continuous integration (CodePipeline) kicks off unit and functional tests as part of the build process preparing for the test environment. Once the review is complete, the changes are now deployed to the development environment where any applicable regression and end-to-end tests are run before the new code replaces the existing in-service code (test then deploy model). Small teams can decide to follow a source-control branching strategy that makes sense if needed: git-flow, etc. The "test" branch is dedicated to the test environment where as "master" is for production

FlowTrack practices continuous delivery of code into production through multiple environments: test and production. The deploy process and infrastructure roll-out are written as code (using technologies such as AWS Cloudformation) and managed under source control.

FlowTrack’s multiple lower environments (local dev, test) provide an ecosystem of sample data sets that exercise the application and services when test automation is run. The test environment is where the system may be stressed for performance and scalability. Performance and scalability changes are driven by metric data captured through monitoring and logging (metrics before and after change – typically captured as part of the issue description/writeup).

Deployments to production are gated by change control process where an issue is opened which identify what is new/changed (Jira). Sign-offs are recorded by development, testing, security, and product management roles which may be one person. Production roll-outs happen on a regular basis without impact to service. This continuous process allows for security updates to roll out regularly and with urgency. If there is impact to production, a rollback is performed to restore service and whatever caused the problem is reverted from source. This restarts the re-proposal approval process of source changes or when possible may be done directly in AWS deploying the previously approved version. This process keeps the set of differences between the test environment and the production environment as low as possible.

In the continuous delivery mindset, features are not released by the deployment of code into production, instead features are enabled in production at the appropriate time (dark launching). Feature toggle enablement in production is gated by a change control ticket (Jira) that follows the software roll-out approval process. Feature or permission toggle enablement in production can have a few more dependencies than code. Those dependencies may include things like external documentation, early access programs, and internal processes for supporting the feature.

Detailed process and procedures for code promotion and production release: See Configuration and Change Management.

Secure Development Standards

Traceability of code changes allow for our software to be a living entity. Our current system for documenting changes is Jira. Every commit and/or Pull-Request, should have a Jira supplied that describes contextually why this change is necessary and reasonable. These artifacts over time allow for one to trace the lineage of why our production software and services change over time.

All FlowTrack git repositories have a company standard configuration from a AWS CodeCommit perspective. This standard is a guideline and can be relaxed, but socialize when those exceptions are needed.

NOTE: Any Sandbox or early development only projects (and repos) do not follow this standard. And certain projects might be excluded.

Developers follow the branch strategy and code review process below:

  1. All development uses feature branches based on the main branch used for the current release. Any changes required for a new feature or defect fix are committed to that feature branch.

    • These changes must be covered under 1) a unit test where possible, or 2) integration tests.
    • Integration tests are required if unit tests cannot reliably exercise all facets of the change.

    During periods of rapid development releases may be based on a Continuous Integration model where each commit is a released in the code pipeline and is not preceded by a pull request. You will be informed what release model the company is currently operating under

  2. Developers are strongly encouraged to follow the commit message conventions suggested by GitHub and at least reference the title of the ticket your commit is for.

    • Commit messages should be wrapped to 72 characters.
    • Commit messages should be written in the present tense. This convention matches up with commit messages generated by commands like git merge and git revert.
    • Additionally, the commit messages should start with the Jira Issue ID when applicable.
  3. Once the feature and corresponding tests are complete, a pull request (PR) will be created using the AWS CodeCommit web interface. The pull request should indicate which feature or defect is being addressed and should provide a high-level description of the changes made.

    PR are not implemented in rapid development Continuous Integration mode at discretion of management

  4. Code reviews are performed as part of the pull request procedure and/o possibly during the build and code security analysis process. Once a change is ready for review, the author(s) will notify other engineers using an appropriate mechanism, typically by adding reviewers as PR approvers.

    • Other engineers will review the changes, using the guidelines above.
    • Engineers should note all potential issues with the code; it is the responsibility of the author(s) to address those issues or explain why they are not applicable.
    • If changes/commits are made to a PR, it should reset previous approvals and require review and approvals again before the PR can be merged.
    • Once the review process finishes, each reviewer should approve the PR, at which point the original author(s) may merge their change into the main branch (i.e. master).
    • PR can only be merged with at least one approval from a reviewer other than the author.
  5. If the feature or defect interacts with sensitive data, or controls access to sensitive data, or contains security/risky infrastructure changes to the target environment, the code changes must be reviewed by the Security team before the feature is marked as complete.

    • This review must include a security analysis for potential vulnerabilities such as those listed in the OWASP Top 10.
    • This review must also verify that any actions performed by authenticated users will generate appropriate audit log entries.

Source Code Management

FlowTrack development/engineering team uses AWS CodeCommit for source code management. Access to AWS CodeCommit and its configuration standards include:

  • All developers must authenticate to gain access to AWS CodeCommit and code repos hosted on AWS CodeCommit according to standards and procedures defined in the Access Policy:

    • Access control to the AWS CodeCommit web interface must be enabled, via SSO and/or MFA if applicable

    • SSH public/private key access may be used for command line or git access to the code repos

  • All code repos in AWS CodeCommit follow these configuration standards:

    • All repos must have an owner identified and listed

    • All repos are by default private

  • Certain branch access restrictions are enabled

  • Certain pull request (PR) requirements are enforced before merging, including:

    • Must have at least 1 review approval to merge

    • all PR tasks must be completed, if applicable

High Level Application Security Requirements

All FlowTrack software must be developed to include the following general application security principles and requirements. Web applications must also protect itself against the OWASP Top 10 vulnerabilities.

  1. Protect sensitive customer data such as PHI, PII and account passwords. Encrypt data stored (at rest).

  2. Secure data in transit and customer communications via TLS.

  3. Provision strong access control (authentication and authorization). Prevent and report unauthorized access.

  4. Log all transactions and activities to be able to tell who did what, when, where, and how. Mask or remove sensitive data in logs.

  5. Implement client security at application endpoints (e.g. browser, mobile app).

  6. Communicate securely across application endpoints and between service consumers/producers.

  7. Use secure defaults to ensure security when in all error conditions.

  8. Check and maintain the security of all third party and open source libraries/components/dependencies.

  9. Validate all data inputs; encode data outputs when appropriate.

  10. Deploy and configure applications securely to production.

  11. Perform regular vulnerability analysis and apply security patches promptly.

  12. Secure privileged access to production environments and ensure ongoing application monitoring.

All software code must complete a set of security scans/testing prior to being deployed to production, including static application security testing, as well as periodic penetration testing.

Pre-production testing is performed with nonproduction data in nonproduction environments. Health checks are performed regularly or automated in production.

Software vulnerability identified through any of the above processes shall be reported and tracked following FlowTrack Vulnerability Management process as defined in the Vulnerability Management Policy and Procedures.

Secure Design and Application Threat Modeling

FlowTrack Security Team in collaboration with development team performs full Application Threat Modeling and Risk Assessment on per-application basis using a custom approach that relies on industry standards and best practices.

Major application updates are captured via an RFC (Request For Comment) process. The RFC template includes Security Consideration as a required section. This section is used to document abuse cases including:

  • risks identified,
  • attack vectors, and
  • mitigating controls.

Each RFC is required to capture sufficient details of the feature/component to be developed, including use cases, motivation and outcome, and the following design details as applicable:

  • authentication/authorization mechanisms,
  • network communications,
  • data encryption,
  • cloud services used,
  • logging/auditing,
  • data flow diagram/description,
  • edge cases, drawbacks, and alternatives.

The RFC must be approved prior to implementation. Security team is included in RFC reviews via the pull request or ticket process.

Platform Design and DevOps Security Details

Documentation on the FlowTrack internal information stores, tickets, or Wiki may include additional security specifications as well as the security design and implementation details of the FlowTrack Platform and its supporting operations.

Access Control of the Application (Identification, Authentication, Authorization, Accounting)

FlowTrack external software application that is customer facing with access to customer specific data, including sensitive information such as PII and ePHI, implements strong access control, covering the Identification, Authentication, Authorization, and Accounting/Auditing (IAAA) of access and user activity.

The implementation ensures that

  • the user requesting access is the one claimed (Identification and Authentication);

  • only users authorized to access specific data (such as ePHI) are allowed to (Authorization); and

  • their access activities are logs (Accounting/Auditing) according to the FlowTrack auditing standards.

The current implementation leverages AWS IAM and DaaS for user identity management and access.

The backend platform implements granular Attribute-Based Access Control (ABAC) for granting access to specific services and data based on the attribute(s) of a principal (i.e. user requesting access -- an attribute could be the role or group membership or organization the user belongs to) and the attribute(s) of the requested resource (i.e. data or service -- an attribute could be the project this data belongs to).

Free and Open Source Software (FOSS) Security

FlowTrack security and development team implemented a process to

  • Inventory all software dependencies;
  • Scan software dependencies for known security vulnerability;
  • Fix any and all high risk findings prior to production; and
  • Review and identify licensing issues of the 3rd party software and libraries.

The current tool in use is Snyk. Snyk supports Node.js, Ruby, and Java. Documentation can be found at the Snyk website.

Snyk also enables automatic license analysis for dependencies, and this functionality is embedded into the securityScan() CodePipeline pipeline step.

Snyk can be leveraged in local development environment to scan locally and/or at every build for dev to include dev dependencies.

Static Application Security Testing (SAST)

Tools:

  • The SonarCloud Platform - static and dynamic code analysis. Together with the FOSS security tool, the SAST testing was made available to use via the automated security-scan docker image, as well as integrated into FlowTrack CodePipeline pipeline library via the securityScan() step

  • Manual code security reviews are periodically performed by the Security team as follows:

    • Periodic review as part of weekly activities

    • Every pull request which includes security team as approvers

    • By request - Jira issue created on the Security project/board

Dynamic Application Security Testing (DAST)

Dynamic testing is available by request to the Security team and is performed using OWASP ZAP. Security team is currently looking into automation of this type of scanning.

Additionally, manual baseline dynamic testing is available by request to the Security team.

Integration of Dynamic testing into automatic tests

Upon availability of automatic UI tests for applications being developed internally, an extension based on OWASP ZAP is used to transparently work with any UI tests available and automatically add basic dynamic tests into build pipeline.

Penetration Testing

External Penetration Testing

An external penetration testing is performed at least once a year by a qualified security researcher / ethical hacker, on the security team internally, and/or with an external security consulting firm.

Responsible Bug Disclore Program

Public Vulnerability Disclosure

All technology contains bugs, including both functional defects and security vulnerability.

Please report security vulnerabilities to [email protected]

Outsourced Software Development

FlowTrack requires all outsourced software development to follow the same rigor and process as internal engineering. Outsourced developers must develop in our secure environment, accept and follow our security policies and procedures, and comply with the same secure coding standards, including:

  • Receive annual or more frequent OWASP or equivalent secure coding training from sites such as; https://owasp.org/www-project-top-ten/ https://owasp-academy.teachable.com/courses

  • Follow the same source control, code review, and security code scanning procedures as defined.

  • Install endpoint compliance agent that checks to make sure firewall, encryption, patching, password policy, screensaver password, and other required protection is properly configured.

Additionally, the third party firm providing outsourced development services must demonstrate that they have conducted the appropriate screening during hiring.

HIPAA Best Practices for Software Development

Use only HIPAA eligible services in the Cloud (AWS)

Because we use the services provided by AWS for our production environment with contains electronic protected health information (ePHI), AWS is a "business associate" of ours. It is required by HIPAA for FlowTrack and AWS to enter into a "business associate agreement" (BAA).

Our fully executed BAA with AWS can be found on the internal document repository.

We must only used HIPAA-eligible services covered under the BAA to process and store ePHI. Non-eligible services may be used in support of our cloud infrastructure as long as it does not have access to ePHI.

A list of HIPAA eligible services can be found here. AWS regularly updates its services to meet HIPAA compliance requirements. Check the page once in a while to find out if new services have been added.

Additional References:

Separate access and data between prod and non-prod accounts

It is a compliance requirement of multiple regulations to ensure separation of duties between production and non-production environments, including both access and data. Additionally, we should not use production data for dev or test, unless the data has been properly sanitized/masked.

Examples of regulations and certifications that have this explicit requirement include HIPAA, SAS70/SSAE-17, SOC, PCI, and ISO 27001/27002, many of which are on our target list to be compliant with or certified to.

Do not log ePHI

Not only it is a security best practice to avoid sensitive data such as ePHI in application logging, it may be a contract violation (per AWS BAA) to do so. We must not send any ePHI to non HIPAA eligible services in AWS, such as CloudTrail.

Include the right language in notices

Specific language must be included in terms, consents, and notices. For example, we must collect email addresses from patients when they sign up for the PHC, and specify in the terms that we may use the email address provided as a formal method of communication, including breach notification, should a breach occurs that impact their PHI.

Data protection

Follow the requirements listed in the following documents:

Risk Analysis and Compliance Assessment

Read the latest assessment report for more details on FlowTrack's HIPAA compliance.

Production System Monitoring and Paging

Software and systems deployed in production are monitored 24/7 for health check and other major/critical error conditions. The on call team is paged via PagerDuty in the event an error or failure is detected.

Notifications via additional channels such as email are also configured.