eRecon Software Development Case Study: HCA Project Management

W16110
ERECON SOFTWARE DEVELOPMENT AT HOSPITAL CORPORATION
OF AMERICA
Leanne Rayo wrote this under the supervision of Kenneth J. Klassen, solely to provide material for class discussion. The authors do
not intend to illustrate either effective or ineffective handling of a managerial situation. The authors may have disguised certain names
and other identifying information to protect confidentiality.
This publication may not be transmitted, photocopied, digitized or otherwise reproduced in any form or by any means without the
permission of the copyright holder. Reproduction of this material is not covered under authorization by any reproduction rights
organization. To order copies or request permission to reproduce materials, contact Ivey Publishing, Ivey Business School, Western
University, London, Ontario, Canada, N6G 0N1; (t) 519.661.3208; (e) [email protected]; www.iveycases.com.
Copyright © 2016, Richard Ivey School of Business Foundation
Version: 2016-02-08
At 10:00 a.m. on July 28, 2014, the eRecon technical team at Physician Services Group (PSG) in the
Hospital Corporation of America (HCA) gathered in the main conference room, just as it did every Monday
morning. The goal of the meeting was to review the project plan and update task completion for the team’s
two-year undertaking — a six-phase, in-house development of a custom web-based accounting software
called eRecon. On this morning, as the team members began to walk through each task and update the
percentage completed to date, they grew increasingly concerned about obtaining critical source data that
they required from the human resources (HR) department. They were approaching the deadline for that
task, and it was unclear what the consequences would be of not obtaining the data on time and what impact
the delay would have on the next phase’s completion deadline of December 8, 2014. There was a large
degree of uncertainty surrounding the process of completing the task.
The project managers, Frank Lafferty and Leslie Ray, arranged a meeting to analyze the project plan and
promised to report back to the team with the results of their analysis. It was crucial that they determine how
the delay in obtaining HR data would affect the project and the 125 end users who were patiently waiting
on eRecon’s completion.
HCA AND PSG
HCA, a publicly traded company founded by physicians in Nashville, Tennessee in 1968, was one of the
largest healthcare companies in the United States. HCA owned and managed hospitals, surgery centres, and
physician offices in 20 states across the country. Due to numerous mergers and transactions, it became a
publicly traded company on three separate occasions between 1969 and 2011. As of 2014, the company
was traded on the New York Stock Exchange with the ticker “HCA.”1 In 2013, HCA reported US$34.2
1
HCA Management Services, L.P., “Our
http://hcahealthcare.com/about/our-history.dot.
History,”
HCA
Healthcare,
accessed
December
6,
2015,
Page 2
9B16D002
billion in revenues and a net income of US$1.56 billion (see Exhibit 1).2 HCA was comprised of over 2,000
subsidiaries, one of which was Physician Services Group (PSG).
PSG employed approximately 13,000 people and was designed to bring business solutions to medical
providers through ownership or management of physician’s offices, giving doctors the ability to focus on
the most important part of their work: treating patients.
HCA was patient focused, and the culture of the organization (including its subsidiaries) was primarily
driven by principles of ethics and compliance. One of the most pertinent laws to every healthcare company
in the United States was the Health Insurance Portability and Accountability Act (HIPAA) privacy law.3
The rules established within this law were carefully followed in every business decision made by employees
at each level of the organization, without exception. Every employee was trained annually on the privacy
law. A major component of the law was a directive on the use of protected health information (PHI). PHI
included identifiable demographic and genetic information, among other personal details relating to the
physical or mental health of an individual, health care provided to that individual, and any payments
received from the individual.
HCA implemented an information protection program in order to enforce and educate employees on the
privacy law. As part of this program, employees were emailed informational bulletins on a regular basis
that focused on one specific issue at a time. All employees, whether corporate or hospital based, were
responsible for the protection of PHI and any other information that could be considered confidential (e.g.,
social security numbers).
PROJECT MANAGEMENT FOR SOFTWARE DEVELOPMENT
Lafferty and Ray were both educated in project management and proficient in the use of Microsoft Project.
However, neither one had any prior practical experience in project management for software development.
To ensure that they had a handle on creating the project plan, Ray had reached out to a more experienced
colleague in the information technology department at the beginning of Phase 1, six months earlier in
January 2014. In those meetings, Ray had discovered a few key details. Software developers were generally
resistant to providing time estimates on tasks. The industry standard was to triple developers’ estimated
schedule time4. Recommended best practice was to never schedule a task’s duration for less than one day;
any tasks estimated at less than one day were rounded up.
Managing a software project was a complicated endeavour. Before anything could begin, project managers
and the team of individuals involved in the project had to first agree on what approach they would take for
the software development life cycle (SDLC). The SDLC outlined a series of steps that were necessary for
the process of creating an application or software. Most commonly, these steps included analyzing,
defining, designing, planning, building, testing, and deploying. There were various different types of
SDLCs; the two most popular were the “waterfall” and the “agile” methods.
The waterfall approach to software development was a traditional methodology that identified the various
process steps as discrete and to be completed in logical order. Projects using the waterfall method were
carefully and extensively planned. It was imperative that all parties strictly followed the plan. Testing
occurred once all coding was complete.
2
All figures in US$ unless otherwise stated; US$1 = CA$1.08 on July 28, 2014.
U.S. Department of Health and Human Services, “Rules and Regulations 45 CFR Parts 160 and 164,” Federal Register Vol.
78, no. 1, January 25, 2013, accessed December 6, 2015, www.gpo.gov/fdsys/pkg/FR-2013-01-25/pdf/2013-01073.pdf.
4
The durations pertinent to this case have already been calculated accordingly.
3
Page 3
9B16D002
In contrast, the agile approach to software development was an incremental design process. Rather than
following discrete process steps in sequential order, agile developers worked in increments, coding small
portions of the application and testing each part along the way. End users had several opportunities to
provide feedback and change their requirements. Unlike with the waterfall approach, planning was much
less extensive when using this method. Agile development was ideal for a project that required rapid
production but lacked a specific vision with respect to the final product.
The executive steering committee selected the waterfall method for the development of the eRecon project. A
team of 10 individuals, known as the business team, worked with Ray to define requirements for each phase of
the application. Once they signed off, the technical team worked on designing the application and discussed the
various technical elements. Following the design discussions, the gruelling process of obtaining schedule
estimates from the developers began. After building the project plan, the developers would begin coding the
application on a development server. Once a set of code was ready, the work could be published (or pushed) to
a quality assurance (QA) server to be tested. Following successful testing, the code would be pushed to a
production server, and the application would be live for the accountants to begin using it.
Corey Young was eRecon’s primary developer on Microsoft’s .NET framework. He was a senior developer
who had been with the company for five years. Samuel Mendoza was the developer responsible for database
management. Mendoza was hired as a developer in the spring of 2014. Both individuals brought over 10
years of experience to the project and were well respected in their field. Although it was evident during
Phase 1 of the project that both men were uncomfortable with the waterfall methodology, Young appeared
especially resistant to the concept. It was unclear to Lafferty and Ray where the resistance was coming
from, but as project managers, their job was to obtain estimates and build a project plan for each phase. The
steering committee depended on their plans to identify the deadlines for completion and effectively
communicate them to the accounting department.
eRECON
eRecon was a web-based application designed for accountants and managers of PSG to prepare, review,
and document balance sheet reconciliations and complete necessary certifications on a monthly basis (see
Exhibit 2). The application created several efficiencies for the accountants at PSG. It eliminated the need
for spreadsheets and manual processes and thereby reduced the chances of errors, omissions, and other
risks. In addition, the entire review and approval process was conducted within the software, so date and
time stamps were recorded and retained for future reference. The system offered real-time status reporting
that significantly improved visibility for managers, allowing them to make better decisions and lead more
effectively. Finally, eRecon included a robust reporting module.
Prior to eRecon, the accountants prepared balance sheet reconciliations in Excel or other tools and printed
the reconciliation cover sheet and all supporting documentation. Each reconciliation was then submitted to
the appropriate reviewer who would review it for accuracy and physically sign the cover sheet. At the end
of each month, all reconciliations were compiled by facility, using the company identification (CoID) code.
A cover sheet summarizing the accounts in each CoID was printed and placed on top of each pile of
reconciliations. The packages were then submitted once more to the appropriate reviewer, who checked to
see that all reconciliations were completed and contained the appropriate level of review. After the final
sign-off, the reconciliation packages were filed for future retrieval in physical filing cabinets. After one
year, the files would be boxed up and sent for storage at Iron Mountain, a secure records management
company, where they were retained for seven years.
Page 4
9B16D002
Initial discussions pertaining to the eRecon project began as far back as November 2012. However, the
team did not begin documenting requirements until October 2013. Around that same time, three Structured
Query Language (SQL) servers were purchased at a cost of approximately $6,000 each. The three servers
were to be used as the development, QA, and production machines. They would house the back end of the
application, which included databases, tables, and stored procedures (SPROCS) that were critical for
providing data to the front end of the application (e.g., general ledger balances and user names), and
capturing and archiving data entered on the front end (e.g., reconciling items and review notes). Initial
estimates put the project on a two-year completion deadline. The project was divided into six distinct phases
to more effectively manage the development and rollout to the 125 potential users.
Phase 1 was denoted “global approvals.” This phase included the creation of the project’s home page and the
global approvals page, which was only accessible by directors and the vice-president (VP) of financial
services. The concept of global approvals was exciting and unique to PSG. Effectively, management figured
out a way to reconcile and review over 12,000 of the 35,000 general ledger accounts with the click of a button
that still satisfied internal and external audit requirements (see Exhibit 3). To do this, low-risk groups of
accounts were identified by management. Periodically, management reviewed the current groups and added
more groups of accounts that were determined to be low risk. As long as a source document was available and
could be generated electronically, the account was eligible to be classified as globally eligible (i.e., low risk).
Every month, each low-risk account would be tested (in the back end of the software) to determine if it
qualified for global approval for that fiscal period or whether it would have to be traditionally reconciled.
The software checked a number of criteria, most of which were related directly to HCA accounting
practices. For example, the account could not have a variance between the general ledger (GL) balance and
the source balance.
Global approvals were completed by the VP on the 11th day of each month, the month-end-close day of
the company. At PSG, the financial books for the prior month were closed in 10 calendar days. Therefore,
by the 11th day, the GL account balances were final; no further transactions (i.e., journal entries) would
occur after that day.
Phase 2 of eRecon impacted the larger audience of end users and was labelled “reconciliation and filter.”
The goal of this phase was to create one additional page called reconciliation. This page would include
three primary sections on one screen (see Exhibit 4). The section on the top left was where the user chose
the filter, the top right contained the filter results, and the bottom displayed the reconciliation. The filter
allowed users the flexibility to navigate to their assigned work by populating a list of CoID accounts based
on the parameters set by the user.
Each filter selection impacted what options would be available in the remaining filter fields. For example, if
users selected their own name, only the CoIDs and accounts assigned to that user in the database would be
populated in the CoID and account filters. The filter results listed records containing the CoID number,
account number, and GL balance. The number of records returned was based on the query executed by the
“apply filter” button. Although only nine records were displayable in the results area at one time, users could
scroll to see more rows. A total record count located below the results gave users an indication of how many
records were returned by their filter. Filter results were sortable by clicking the column headers. The
reconciliation section displayed detail of a reconciliation (which was prepared at the CoID account level).
Header information, including the account number, name, period end (PE) date, and GL and source
balances, was populated by the system. A details button located in the header opened a pop-up window that
included requirements for that account, according to company policy, and spaces for users to enter helpful
Page 5
9B16D002
information and key contacts. Beneath the reconciliation header was an area for users to add reconciling
items and to conduct their reconciliation.
Each reconciliation item line included a PE date, description, amount, and note field. The amount was
critical to calculating the unreconciled variance. The unreconciled variance was required to be zero in order
for a CoID account to be considered completely reconciled. The formula for the unreconciled variance was
as follows:
Unreconciled Variance
Variance
Sum of Reconciling Items
where,
Variance
GL Balance
Source Balance
In this phase, users only had the ability to save or print data. Workflow and approvals, however, were not
within the scope of this phase.
Phase 3 was intended to replace an existing reconciliation tool called the “post close review guide” (PCRG),
so this phase was known as the PCRG. It was the biggest phase in terms of user interface development. The
PCRG was a Microsoft Access application (both front end and back end), primarily used for recording
completion and approval of reconciliations and reporting. In this phase of eRecon, enhancements to the
reconciliation page were necessary, in addition to the creation of two new pages: PCRG and “monthly
financial certification.” Users were able to prepare reconciliations and submit them for review. Reviewers
subsequently reviewed and then chose to approve or reject the reconciliations with notes explaining what
actions were needed.
Once preparers responded to those notes and resubmitted them, reviewers checked the reconciliations again
and then approved them. At this point, the CoID account was considered completed. PCRG was a review
of the balance sheet accounts at a CoID level; this could not happen until all accounts that rolled up to a
CoID were approved. Once all CoIDs in a division were approved on the PCRG screen, managers were
able to perform reviews at a division level on the monthly financial certification page. Upon the directors
signing off on every division, the entire workload within the application was considered complete for the
month, and it was locked to any further changes (see Exhibit 2).
Phase 4 of the application development was called the “support store.” This phase included adding
functionality to the reconciliation page, allowing preparers to attach documents to reconciliations as
supporting documentation. Reviewers would then have everything they needed in one place to approve the
reconciliation.
Phase 5 was called “journal entries.” For this phase, the business team wanted the ability to create a journal
entry directly from a reconciliation item line. Just by specifying that they wanted to create an entry and then
entering the offsetting CoID and account, they would let the system know to populate the two-line record
and upload it to the general ledger, once approval was received.
Phase 6 was classified as the “home page and enhancements” phase. The home page, already created in Phase
1, was enhanced to include a dashboard that consisted of an individualized summary of a user’s assigned
accounts with corresponding status of each account. One example of the enhancements written into this phase
included automatically attaching supporting documentation to the support store for some accounts.
Page 6
9B16D002
PHASE 2 — RECONCILIATION AND FILTER
Phase 1 of eRecon, global approvals, was completed on June 6, 2014, when it was successfully rolled out
to production. The VP of finance and the directors were the only end users impacted by the rollout. The
next working day, development of Phase 2 began. There were two primary components to this phase: (1)
SQL server database setup and (2) Microsoft .NET user interface development. Mendoza was assigned
primary responsibility for SQL database tasks, and Young was in charge of the user interface. Young was
assigned to the eRecon project on a full-time basis, while Mendoza worked on eRecon-related tasks for
approximately two-thirds of his work week.
SQL Server Database Setup
The SQL server database setup was critical to Phase 2 of eRecon development. Before Young could push
the application’s user interface to QA for testing, Mendoza had to complete the “shared” and “security”
databases. Although it was important that these two databases were both completed prior to eRecon, the
actual order in which they were completed did not matter. The team set a goal of completing one of the
databases by the end of August and the other database by the end of September, so Mendoza decided to
work on the shared database first.
Shared Database
The shared database contained four main tables: “company master,” “account balances,” “general ledger
account list,” and “journal entry transactions.” For each of these tables, it was necessary to prepare the
source data (two days), develop the optimal load (two days), and finally load and validate the data in the
development environment (two days). Once the four tables had been validated, they would all be pushed to
QA for testing at the same time. Pushing the shared database to QA would require five or six days, which
included creating and submitting a work ticket to the systems group in the information technology and
services (IT&S) department (one day), pushing to QA (one to two days), validating performance and data
(two days), and verifying the load (one day). The tasks had to happen sequentially.
The IT&S department was responsible for pushing to QA. Essentially, this meant that Mendoza provided these
workers with scripts as a part of his work ticket that they would then execute on the QA server. Segregating
duties in this fashion was important for change control, which was enforced by audit. IT&S scheduled all pushes
for Tuesdays only. Although only one day was actually necessary to complete the push, it could take up to two
days for the systems group to execute the scripts, depending on its current workload. It was not uncommon for
the technical team’s requests to get delayed by one day because eRecon was not considered a high-priority
application by IT&S. While Mendoza was assigned to validation, best practices called for a separate individual
to verify data. For the shared database, Mendoza’s manager, Michael Craft, would accept responsibility for
verifying the data. Craft was available to work on the project for 20 per cent of his work week.
Once the database was tested in the QA environment, it could be pushed to production. The tasks included
in this push were the same tasks required to push to QA, so it would take the same amount of time (five or
six days). At the July 28 project meeting, the team confirmed that the shared database was 100 per cent
complete, and they were ready to begin work on the security database.
Page 7
9B16D002
Security Database
The Security database included two large tables — “employee information” and “employee fact” — in
addition to several other small tables, categorized as system or base tables for the purposes of project
planning. However, before any work could begin on the tables, a few items needed to be addressed. First,
the technical team, which included Lafferty, Ray, Young, Mendoza, Craft, and Daniel Mayer, needed to
finalize table-naming conventions. They could do this in one day. Second, Mendoza would merge the
existing security tables. He also needed one day to do this. In the meantime, Young needed to gather the
minimum technical requirements for the security database. This was specific to the filters section of eRecon
and was not expected to take longer than one day. Once requirements were gathered and the tables were
merged, Mendoza would create the views and SPROCS necessary for eRecon to operate properly. This
could be completed in three days.
The process for completing the two primary security tables (employee information and employee fact), plus
the system or base tables, was the same for each one. It did not matter in what order these tables were
completed. The first step for each table was to identify, obtain permission for, and extract the pertinent raw
data from the appropriate sources. Mendoza was responsible for completing this for the employee fact table
and for the system or base tables, which would take two days each to complete. Daniel Mayer, one of the
managers on the technical team, was responsible for getting the raw data from HR for the employee
information table. He was committed to the project for 50 per cent of his time, and was a valuable asset to
the team overall; the group was relying on him to obtain this data from HR.
At the time of the July 28 meeting, Mayer reiterated that the task was more complicated than it appeared
because the team would need to work through a significant amount of bureaucracy to gain access to the
system that contained the information. Mayer had started working on it several months earlier to get a head
start, but even with the approval of the chief executive officer, it was not yet complete. He was confident,
however, that if he went back at it by the next Monday (August 4), he would have access to the database
with the source data within 10 to 15 working days. After obtaining the raw data for each table, the technical
team was then responsible for defining the scope and the business rules of the load. At minimum, this would
take two days for each table but could take as long as one full work week.
Software called SQL Server Integration Services (SSIS) was being used for all extract, transform, and load
processes. This was necessary to load the millions of records of data that were required into the security
and shared database tables. The next step in completing the security tables was to build, validate, and test
the SSIS packages. This would take Mendoza three days per table. Once all packages were tested, a push
to QA could occur. The tasks would be exactly the same as the push to QA for the shared database. Once
that was completed, the content could be pushed to production. The steps for this process looked identical
to the shared database push to production. Once it was pushed to production and validated, the security
database would be considered successfully completed.
Microsoft .NET User Interface Development
One of the most exciting properties of the eRecon application was the web-based front end. Applications
with a Microsoft .NET front end were more appealing to users because they provided a more sophisticated
user experience than other platforms (e.g., Microsoft Access) and gave developers more flexibility in
design. Overall, they were more appealing and better functioning. The user interface would be primarily
completed by Young. For purposes of project planning, he organized the development work into five
sections: interface, account grouping, batch process, security, and archive. After the five sections were
Page 8
9B16D002
complete, a push to QA would occur, with at least one month for user acceptance testing, before the
application could be rolled out to the production environment for the end users.
Young decided to create the front end using the model-view-controller (MVC) design pattern, which was
commonly used by software engineers. For eRecon, the controller was a web browser, the model was
Hypertext Markup Language (HTML), and the view was SQL code. In order to complete the MVC for the
interface section, Young needed between eight and 12 days but could not begin until the security database was
complete. Once that database was completed, he would need to persist with the data (four days) and complete
the client scripting (10 to 15 days). Unit testing would then be complete. Young required a minimum of five
days and a maximum of seven days to complete it for the interface section of development.
To complete the account grouping section (which could begin as soon as the shared database was complete),
Young needed 10 to 12 days. The batch processing section would be completed by Mendoza, and it did not
require any prior work to be completed. This included creating a process for the reconciling item carry over
process and updating the current account creation process. The batch process section would take 10 days to
complete. The security section, which could not be started until the security database was completed, would
entail an MVC that took Young approximately two days, and coding the service and repository, which
would take Young three to five days. Then, unit testing would take 1.25 days for the security section. Young
also needed to complete the archive section, although there were no prior tasks for that. This included
creating the archive table (one day) and modifying the SSIS package to copy or clear data (one day). Once
all five sections were complete, the push to QA could begin (using the same steps as in previous pushes).
The push to QA required Young to submit a work ticket to the systems group in IT&S (one day). On the next
Tuesday, the group would complete the push. Before the month-long user acceptance testing could begin, Ray
needed to produce the test plan and test scripts, which would be distributed to the testers. This would take her
one to two weeks to complete. She could begin working on this as soon as Young submitted the work ticket to
push the application to QA. Ray was in charge of coordinating the user acceptance testing during the month,
gathering feedback and addressing security issues. Once the testing was completed, the application’s front end
could be pushed to production. The process was similar to the push to QA. After Young submitted a work ticket,
the systems group would push to the production environment (on a Tuesday). The application would be briefly
tested (one day) by Young and Ray and then finally rolled out to the end users.
CONCLUSION
Lafferty and Ray were concerned about Mayer’s ability to obtain the HR data that the team needed to
complete the security database. It was evident that several tasks pertaining to the user interface depended
on this information to be complete. They all believed that it would not be much longer, but as was often the
case in a corporate environment, when working with another department, there were too many uncertainties
to be able to estimate accurately how long a process would take.
The two project managers needed to determine how much slack they actually had built in for this task to be
completed without delaying the Phase 2 finish goal of December 8, 2014. Accounting would begin reconciling
their November balances on December 9. Therefore, even the slightest delay would postpone the rollout to the
next month. Fortunately, Lafferty and Ray knew that they had some flexibility in the development tasks.
Anything other than the actual pushes (to QA and production) and testing could be scheduled in increments and
split up over several different work sessions; they could even be done over a number of days or weeks. The
team’s reputation and future phases of eRecon depended on a successful completion of Phase 2.
Page 9
9B16D002
EXHIBIT 1: INCOME STATEMENT
CONSOLIDATED INCOME STATEMENTS
FOR THE YEARS ENDED DECEMBER 31, 2013, 2012, AND 2011
(US$, except per share data)
Revenues before the provision for doubtful accounts
Provision for doubtful accounts
Revenues
Salaries and benefits
Supplies
Other operating expenses
Electronic health record incentive income
Equity in earnings of affiliates
Depreciation and amortization
Interest expense
Losses (gains) on sales of facilities
Losses on retirement of debt
Legal claim costs
Gain on acquisition of controlling interest in equity investment
Termination of management agreement
2013
$38,040
3,858
34,182
2012
$36,783
3,770
33,013
2011
$32,506
2,824
29,682
15,646
5,970
6,237
(216)
(29)
1,753
1,848
10
17
15,089
5,717
6,048
(336)
(36)
1,679
1,798
(15)
13,440
5,179
5,470
(210)
(258)
1,465
2,037
(142)
481
175
Income before income taxes
Provision for income taxes
Net income
Net income attributable to non-controlling interests
31,236
2,946
950
1,996
440
30,119
2,894
888
2,006
401
(1,522)
181
26,121
3,561
719
2,842
377
Net income attributable to HCA Holdings, Inc.
$1,556
$1,605
$2,465
$3.50
$3.37
$3.65
$3.49
$5.17
$4.97
445,066
461,913
440,178
459,403
476,609
495,943
Per share data:
Basic earnings per share
Diluted earnings per share
Shares used in earnings per share calculations (in
thousands):
Basic
Diluted
Source: HCA Holdings, Inc., “10-K: For the Fiscal Year Ended December 31, 2014” EDGAR Online, accessed December 6, 2015.
Page 10
9B16D002
EXHIBIT 2: eRECON WORKFLOW
Source: Company files.
EXHIBIT 3: GLOBAL APPROVAL SUMMARY
Source: Company files.
Page 11
9B16D002
EXHIBIT 4: RECONCILIATION AND FILTER PAGE
Source: Company files.
EXHIBIT 5: ANNUAL SALARY DATA
Title
Source: Company files.
Starting Annual Salary Range
(in US$)
Project Manager
$60,000 to $70,000
Developer 2
$60,000 to $70,000
Senior Developer
$70,000 to $80,000
Manager
$80,000 to $90,000