|
With the rapid adoption of J2EE has come the realization that more than
just J2EE expertise is needed to successfully develop enterprise
applications.
The following are critical to the success of J2EE projects:
- A well-defined component integration strategy
- The ability to select the right J2EE platform vendor
- Good understanding of internal and external expectations
- Evaluation of technical and nontechnical challenges and risks
- Availability of resources and tools
- Capabilities of the team
- Realizing that the best J2EE design practices have to be tailored to
meet the needs of the specific project
The ultimate goal of any development project is to produce quality
software that meets the requirements of the customer, and is on time and
under budget. J2EE adds another dimension to this challenge. Distributed
Java projects are highly componetized. Componentization has many advantages as well as challenges, the biggest of which is bringing it all together when different components have been built in parallel.
J2EE projects typically combine a variety of components built on EJBs,
servlets, JavaBeans, JSPs, HTML pages, and the database. Each tier of an
application is developed separately and follows a different "life cycle" in
the development process. For example, a minor change in the underlying
database schema or an EJB can cause serious ripple effects in the
application, whereas changing a servlet or a JSP does not constitute a risk
of the same degree.
This article captures our recent experience in successfully managing a
six-month, 22,500 man-hour J2EE project in a startup organization. It
focuses on engineering processes that had to be set up to support the
development of enterprise software. While we borrowed from different
development and design processes, we also had to evolve our own to
custom-fit into our environment, as is normally the case with real-world
projects.
The primary motivation for writing this article was to pen down the
processes, pitfalls, and challenges of successfully completing a fairly
complex J2EE project in a bootstrap environment. We describe a practical
approach to release and change management for a successful J2EE project
including setting up the environment; defining project teams, the tools and
processes used; and J2EE-based design, development, and deployment.
The Environment
It goes without saying that the right level of J2EE expertise is a must
to make a J2EE project successful. Equally important is having the correct
environment for the team to be the most productive. The development
environment typically includes:
An IDE that supports Java development
Configuration management tools and processes that support various
distributed components
A version-control system that supports Java applications
A bug-tracking system
Build processes and an overall release methodology
-The release methodology has to incorporate the fact that business
components are built and released in a different manner than Web components
built using servlets and JSPs.
-The dependencies of the business layer and the presentation layer have
to be taken into consideration in the release process.
The release methodology in our project was customized to support the
multitiered nature of J2EE architecture. The development and release of each
tier was done separately so that a defect in the components produced by one
team would have minimal impact on the rest of the team.
The development teams for this project had a couple of seasoned Java
architects and developers as well as junior engineers. The end game was to
develop a software product that was highly customizable and would adapt to
different industry verticals. BEA WebLogic Server was used as the
application server. Specific J2EE technologies used in this project are
discussed later in this article.
The project consisted of five major independent applications that
reflected real-world functionality. An important objective of the
application suite was to ensure that these components could be run as
independent products as well as an integrated suite. They were developed as
different applications within the same application server.
The development team was split into five subteams; each team had between
two and four developers and was responsible for one application. This turned
out to be the optimum team organization for a company of less than 30
employees.
In all, three different types of environments were set up to support the
release management process (see Figure 1).
Development Environment
The boxes in the left section of Figure 1 depict independent Windows
2000 desktops and laptops. Each developer's personal desktop was used as the
development environment. It had access to a database schema that could be
used for development purposes and was focused on a particular tier of the
application - the JSP and HTML tier, the servlet tier, the EJB and business
component layer, and the database layer. Each developer worked on a specific
layer at a given time.
Integration Environment
The middle section of Figure 1 shows the integration environment. A
single application server machine was set up to run all five applications
within BEA's application server. The application server pointed to a
dedicated database schema reserved for integration testing. Components
developed separately in the development environment were brought together in
the integration environment.
Staging Environment
The staging environment consisted of two application servers and a
database server. This closely resembled the production. Once the application
was integrated and packaged into an installable application, it was promoted
to the staging environment.
Project Life Cycle
The project was broadly divided into a development phase, an integration
phase, and a test phase, as is the norm. However, since we were in startup
mode, it was crucial to expect and accept changes in direction in order to
market and sell the product suite. While testing was started in parallel
with the development phase, it was not until the integration phase that the
entire application was available for testing.
Being a startup organization, an interesting distraction was to support
ongoing sales activities during the entire development phase. Demos had to
be built in short periods of time for the sales team and required the entire
product suite to function in an integrated manner. Such unscheduled
activities can potentially derail a well-planned development project.
Instead of treating the demos as distractions, we used them as opportunities
to develop prototypes for the yet undeveloped features. This enabled the
sales team to present a more complete solution to the prospective client.
Even as the demos proved to be a great advantage, there were many
challenges. The top challenges were:
To ensure that the development effort continued while part of the team
was diverted to build the demo. Existing activities were reprioritized so
that they would have minimal impact on the schedule.
To evaluate new requirements (generated from the demos) and their
impact on the existing design, functionality, and schedule. If major changes
had be made to the underlying data model or in the EJB layer, they were less
likely to make it into the release. On the other hand, if new requirements
could be implemented by changing servlets or JSPs, there was a greater
chance that they would make it into the release.
To roll back ad hoc enhancements into the development cycle.
Due to this continuous feedback, the team was able to redesign certain
components to better reflect the requirements from the field. At least one
engineer from each team was assigned to work on the demo. Putting together
an entire demo was another interesting activity, since it was always a
hybrid of well-designed features as well as those features that were just
prototyped for the demo. J2EE is an ideal platform for managing a changing
codebase because of its inherent support for decoupled components.
J2EE Technology Components
The project conformed to the standard three-tier J2EE architecture. The
application suite was built using the following:
- JSPs, servlets, and JavaBeans formed the presentation layer.
- EJBs encapsulated the business logic.
- JMS was the messaging layer for the asynchronous exchange of business
events.
- BEA WebLogic Server was used as the application server.
- BEA WebLogic Integration was used as the J2EE platform for business
process management.
- Oracle 9.1 database housed the business data.
- Several third-party libraries were deployed to support charts, graphs,
etc.
Change Management
We used Ant-based compile scripts to set up the integration environment.
The staging environment was set up so that the application could be
installed via the installer. Microsoft's Visual Source Safe (VSS) was used
as the version control system.
During the development phase, the integration environment was built on a
regular schedule. Earlier in the project, the frequency of a new build was
once a week. The build frequency increased to 3-4 builds a day toward the
end of the release. The steps involved in rebuilding the integration
environment were:
Check out the latest code base from VSS for the entire product suite
along with all necessary configuration files and incremental schema changes
if applicable.
Build the entire product suite using Ant scripts.
Configure all the necessary properties files.
Apply incremental schema changes to the database.
Since each application could be run as a standalone system, the
following changes had to be made via configuration files to execute them
within the same environment:
By default, each application was set up with its own set of libraries,
JAR files, classpaths, property files, etc. The applications had to be
configured so that they shared a common environment. Thus a common
configuration file had to be created that would be used by all the
applications, and a common "lib" directory had to be created that contained
the shared libraries and JARs.
Application-specific EJBs and JAR files were local to each application.
By default, each application pointed to a different schema. The
properties files had to be updated so applications shared the same database.
Some of the HTML files and JSPs had to be modified to provide an
integrated user interface.
Figure 2 shows how the directory structure was set up to ensure that all
common components were shared. In Figure 2, application 1 has its own set of
classes, JARs, config files, JSPs, etc. It also shares the common classes,
JSPs, JARs, etc., from the Common Environment folder.
Each component was developed and tested in its own environment. Once the
key features were developed, they were integrated in a separate "integration
environment." Once the applications were integrated, they were promoted to
the staging environment. The defects found in this environment were
reproduced and analyzed in the integration environment.
Figures 3 and 4 show the change management steps across various
environments.
Change Management of Shared Components
Shared components implement functionality such as user management,
common logger, and audit trail. Common functionality is shared across many
applications. All EJBs, property files, JSPs, or database entities that
support such common functionality have to be carefully managed since it
impacts more than one application. Additional coordination is necessary to
ensure that other teams can continue to be productive. The changes were
completely tested in the integration environment before checking them into
VSS. In other words, the next steps were followed to ensure the quality of
the shared components:
Obtain the most current code base from VSS.
Obtain the modified shared components from the developer's environment
and overwrite the code base from VSS.
Compile the application using the integrated build script.
If the functionality works as expected, immediately check the modified
components into VSS.
Change Management of the Business Logic Layer
Sometimes changes in the EJB layer meant changing the presentation layer. JSPs and servlets may have to be fixed to reflect new
changes in the EJB layer. Such changes were coordinated by the development
team and implemented in the development environment. Developers working on
the presentation layer used the updated EJB JARs and ensured that the
presentation layer worked. All the changes were then checked in at the same
time. Figure 3 shows the steps involved.
Change Management of the Database Schema
Typically, schema changes made to support new features are most likely
to break the system. Schema changes made to fix defects must ensure that
existing features are not impacted. However, they were discouraged in this
project, and were made only if they impacted critical functionality. Change
management of schema changes is shown in Figure 3.
Change Management of the Presentation Layer
Figure 4 shows the change management steps in the presentation layer. It
can be seen that some changes are rolled forward from the development to the
staging environment and some changes have to be rolled back from the staging
to the development environment. Since the presentation layer has the least
number of dependencies, it can be changed easily in any environment. During
testing, it was sometimes easier to tweak a JSP in the test environment
itself. These changes were then applied to the source in VSS. The challenge
here was to be disciplined and roll the changes back into the development
environment.
Tools and Processes
Various tools and processes have to be in place to support software
development. Some of the key management aspects that must be addressed are
time, process, version control, configuration, and defect. Being a startup
organization, the team had to set up each and every process and tool from
the ground up. The following are some of the criteria used to select these
tools and processes:
- Should adequately support the requirements
- Be easy to use, configure, manage, and measure
- Be cost effective
- Be scalable to support a growing team for at least two years
- Have a relatively low learning curve for the team
- Be practical to use and not create additional work
Configuration Management
Configuration management controls and monitors change throughout the
software development life cycle. Since we had to branch out often to support
ongoing demos, it was extremely important for the team to follow a
consistent configuration management process. Visual Source Safe (VSS) was
used for configuration management and version control. Every element of the
project was under configuration management right from the get-go, including:
- Requirements documents
- Configuration files
- External libraries
- Source code
Defect Tracking
There are several defect-tracking tools available under public domain
that work well in small organizations. A tool called codeCharge code
generator was used to build the bug-tracking tool. CodeCharge generator
allows a developer to quickly create a Web-based application in a variety of
languages including ASP, ASP.net, JSP, and ColdFusion. It supports all
standard relational databases including MS Access. This tool enabled us to
incorporate new features without much down time.
Build Process
Each developer set up his or her own build environment. The solutions
ranged from using the built-in functionality of an IDE to using Ant and the
basic batch script. Developers were not allowed to check in functional
enhancements without ensuring that the entire application compiled and the
new functionality worked in the local environment. A common build script
using Ant was developed for each application. Thus, getting the most current
version was as simple as obtaining the current code base and building it
with the Ant scripts. The build scripts failed very rarely since the
developers validated each and every code change.
Test and QA Process
You rarely have the luxury of a dedicated QA team in a startup
environment. While the development team was busy building the new product,
the domain experts were busy creating a functional test plan. A "war room"
environment was set up for the entire development team for two weeks
following the integration phase. With help from subject matter experts, two
weeks were dedicated to testing the new product. Defects were documented and
classified. Defects that prevented the execution of key functionality were
classified as "show stoppers" and fixed immediately. The remaining defects
were prioritized and either fixed following the test cycle or documented and
scheduled for the next release.
Challenges and Compromises
Embarking on a new J2EE project is always challenging, especially in a
small growing environment where the success or failure of the project
depends on each decision - from selecting the right tools and setting up
processes to making hard decisions and managing expectations. This project
was no different. Some of the project management challenges were:
Requirements Management: Since the sales team was able to provide
feedback well into the development cycle, requirements were never really
frozen. Each new requirement was carefully evaluated and prioritized. If a
new requirement was deemed to be critical for the release, it would be
further analyzed and the impact on the schedule and design evaluated.
Testing and QA: A unique environment involving the entire team had to be created to test the product. The war room environment helped the team focus on the task at hand and iron out the rough edges.
Expectation Management: Since this was a new team and each person brought in unique experiences from prior jobs, managing the team's
expectations was a high priority. The team had to be reminded frequently
that the integration environment was for integration testing, not
development, and that changing requirements have a huge impact on the
quality of the product, etc.
Time to Delivery: As with most development projects today, time-to-market was the most critical metric for this project. Even though it
was challenging, the project was completed on schedule by making certain
compromises.
Mentoring: The project plan had to include mentoring activities, given the varied levels of J2EE expertise in the development team. Since there was a learning curve, productivity metrics had to adjusted for the entire team.
To meet the goal of delivering a quality product on time, several
compromises had to be made. Several performance-related issues were
identified, but not resolved as this would have pushed the project beyond
the release date. Portability was postponed to a future release. Compromises
also had to be made when selecting tools and processes, since the bootstrap
environment did not justifying buying commercial tools such as Rational
Rose, ClearCase, WinRunner, and TestDirector. In some cases, functionality
that should ideally be developed in the EJB layer was developed in the
servlet layer to meet the time-to-market pressures.
Conclusion
J2EE projects can be completed on time by following the basic principles
of project management. They can prove to be more challenging, as they target
enterprise applications that require large integration efforts. Identifying
potential risks and then taking quick actions to mitigate them keeps the
project under control. This prevents small "fires" from destroying the
entire project. Technical issues are relatively easy to resolve as compared
to people and process issues. It's the responsibility of project management
to get a buy-in from the stakeholders on the processes and tools to be used.
It's human nature to resist change, especially when you don't see value
in the process. Using best practices as a guideline and making pragmatic
decisions go a long way in making any project successful. Fortunately, J2EE
is mature enough, so you can learn from other people's experiences.
|
