Draft PBO Proposal
Standing Committee Preliminary Report
To: jdavis
cfa.harvard.edu, wprescottusgs.gov
cc: Paul Segall (segallpangea.Stanford.EDU), Paul Silver (silverdtm.ciw.edu),
"Duncan Agnew" (agnewbilby.ucsd.edu),
"Mark Simons" (simonscaltech.edu), "Sue Owen" (owenterra.usc.edu),
"William Holt" (wholthorizon.ess.sunysb.edu), mikejucar.edu,
shiverucar.edu
Subject: Standing Committee Report on First Draft
Date: Tue, 30 Jul 2002 07:41:14 -0700
From: Frank H Webb (fhwcobra.jpl.nasa.gov)
Will and Jim,
The PBO Standing Committee recognizes the need for early and quick
feedback on the PBO proposal and are submitting this first report
summarizing our initial input on the first draft of the PBO proposal.
While there are issues due to the proposal being in its beginning stages
and several committee members being partially available or unavailable
for comment, this initial report is being forwarded along to UNAVCO,
so that the proposal team can get some initial feedback. The report is
broken into three parts consisting of general comments, specific replies
to the 8 questions posed by Jim Davis, and specific comments on the text.
Also, we would still like clarification on the following questions. I know
that we discussed some of them last week, however, an updated electronic
response would be helpful to document the issues.
1. The proposal schedule - When do you intend to submit it to NSF and
what is the schedule for reviews between now and then?
2. The intended audience - Who is the audience? What is the depth of
detail that they expect? Do they expect the focus to be on science,
management, and/or cost?
3. What is your expectation of the scope and depth of the proposal?
4. Could we get a budget that is broken down by work force?
5. Could we get a master schedule for the tasks involved?
6. What is your expected funding profile?
Sincerely,
Frank
-----------------------------------------------------------------------------
General Comments:
The proposal is considered to be a good start. However, its flow is
somewhat uneven and it could be greatly improved by laying out in the
beginning the science rationale for the observatory and how that rationale
maps into the requirements. The drivers and rationale for the technical
approach, schedule, and management should flow down from these higher
level requirements. These should then be used as the basis on which the
costs are estimated. A plan and schedule on which these cost estimates
are based should be developed, including a high level work break down
structure (WBS) for the proposed effort and better costs justification.
In the context of the bugdet, the proposal team should do what it takes
to preserve the science goals of PBO in this early planning stage. The
budgets should be scrubbed in order to preserve these goals and determine
the level of descope, if any, of the project.
There is a sense that essentially only Mike Jackson is working on
the proposal at what seems to be a best efforts basis. This would
be inadequate given the importance of the proposal and size of the
request. It seems very risky for UNAVCO Inc. to accept what seems to
be a relatively high level of uncertainty in the costs and put out a
proposal. One risk is that they will not make a credible argument for the
project, which could result in it not being funded. And, if it is funded,
it has the risk that UNAVCO will not be able to deliver at the proposed
cost and the first project of the corporation will have to be descoped
or canceled due to poor costing and planning. More resources should
be applied to this very important project, especially, in the area of
planning and cost estimating.
Central control of the implementation seems reasonable given the
complexity of the implementation and is key to controlling the project
and the costs. The plan and processes for how this management will be
responsive to the community input needs to be defined.
The omission of any reference to existing networks, their roles in PBO,
and experiences learned from them is a major weakness.
Specific responses to questions:
1. Evaluate whether the current PBO facility plan, as outlined in the draft
proposal, has a high probability of meeting the community's observational and
science goals for PBO. If not, then please be specific as to why.
No. It proposes a descope in the numbers of GPS and strainmeter
stations stating cost as the reason. The costs are not well
enough documented to determine how credible they are and if
descoping is necessary.
In addition, the decision making process for descoping and
prioritizing is not defined.
The role of the Regional Scientist is confusing and needs to be
better explained.
2. Evaluate whether the proposal is targeted to the right audience, including
geo-scientists but also scientists from other disciplines and educated
non-scientists (such as members of the National Science Board). Are there
technical terms and concepts that need further explanation or that could be
omitted without loss of meaning?
No. The proposal needs to address how the PBO will help to
answer significant questions. The link needs to made as explicit
as possible.
There ought to be a page or two (as an appendix) on what
these systems are, how they work, and what they measure--and
probably another page or two on what a plate boundary is and
what happens there. The geoscientists will skip it, others
will be grateful. Think Scientific American level. The focus
of the text on monumentation will seem bizarre to some readers,
and needs to be justified.
As written, the flow of the text is somewhat uneven in that it
has in some places too much detail and others too little.
3. Evaluate whether the PBO plan is flexible enough to allow lessons learned
from the early years of operation to be factored into later implementation
plans.
There is not enough detail to make this assessment. As a
related note, the proposal does not indicate how it has applied
the lessons learned from previous network installations to this
project. The relevance of these lessons and the costs associated
with the tasks in the other projects should be assess and compared
with the existing project. For what appears to be a relatively
complex deployment and operation for implementation, it is not
sufficient to merely assert that the costs are understood and
present average station costs without some data or justification
on the assumptions that go into those costs.
There is concern about rigidity from the regional boundaries.
Given that something like this is unavoidable, it would be
preferred that these followed tectonic boundaries rather than
state lines (perhaps a tectonic approximation to state lines?).
4. Evaluate whether the allocation of resources is optimal for the PBO science
goals, e.g., number of continuous GPS stations vs. borehole strainmeters, or
number of backbone GPS stations vs. clusters. Does the proposed facility
address the highest priority plate boundary zone questions?
There is not enough detail to make this assessment and in some
sense it is late to be raising these questions. On Some Pre-PBO
funds were allocated to address this and simulation studies
may be performed in August to try and optimize PBO station
deployment. Unfortunately this should have been done a long
time ago.
Additionally, we may never really know until afterwards. Station
distribution will constantly evolve as sites are permitted and
deployed, and areas of interest are found to be too expensive
or impossible to build in. In the proposal, the point should be
made that the community has spent a lot of effort to try to make
this the case--several well-attended meetings. So it is pretty
close to what we think is optimal, whether it actually is or not.
5. Evaluate whether the data archiving and data products plans are adequate to
meet the needs of individual investigators. If not, how could it be
strengthened?
In general, yes. Though, the assumptions behind the proposed
plans should be explicitly stated. How many individual
investigators are expected? What products do they want? Raw
GPS? Tropospheres? Station velocities? How will they interact
with the system? Interactively via the Web or from the command
line in some automated script?
There was a concern that the need for two archive centers
was not justified. The availability and cost of existing
commercial capability for mirroring big datasets at separate
sites should be considered as an alternative. Nothing we do
is "mission-critical", hence we can afford to have occasional
1-day outages, also--except right after an earthquake (or the
week before AGU). This, and the fact that most of the data do
not change once collected, ought to make the task of getting
backup cheaper.
6. Evaluate whether the implementation schedule is realistic and
achievable.
No. The schedule is too simplisitic and optimistic for the
reconnaissance, permitting and deployment. Even though permitting
is flagged as having great uncertainty, the proposal gives the
sense that it will be easily dealt with and that the capability
to construct sites will control the schedule (and implicitly
the costs). This is at odds with the lessons learned from
previous deployments of large GPS networks. More detail needs
to be provided to justify why the reconnaissance and permitting
will not be the critical path in the implementation and why this
will not impact the average cost per site.
7. Evaluate whether the staffing levels are realistic and sufficient to
accomplish the science goals of the PBO facility.
May be not. The staffing levels are reported in $ and not
level of effort or FTEs and are not linked to a work breakdown
structure (WBS) that clearly shows the tasks and the workforce
and other direct costs (ODCs) required. This suggests that the
staffing levels are in part driven by trying to meet a cost cap
and were not estimated based on the level of effort required.
More justification of the costs would make it easier to evaluate
the costs.
There is some concern that the borehole strainmeter effort has
a lower level of staffing. There is an 8:1 ratio of numbers of
sites, but I think this may be more than counter balanced by the
time needed at each site and the greater involvement that will
probably be required of PBO staff in the process. The relative
levels at least need to have serious thought, as part of the
overall budget planning.
In addition, the budget contains no administrative staff for
staff support. Some dedicated staff will be probably be needed
for general support, processing shipping paper work on all of
the equipement that will flow through the facility, site permits,
data entry, etc.
8. Advise as to whether there should be a further reduction in the number
of GPS and/or strainmeter stations in order to create a 10% management
reserve to meet future contingencies.
The need for this level of reserve is not argued in the proposal.
Some reserve is certainly necessary given the high uncertainty
in a number of the tasks and activities. The reserve strategy
and the proper level of reserve need to be argued and justified,
and applied at the WBS element level.
If the reserve strategy is to reduce the number of sites this
needs to be spelled out since it will have an impact on the
scientific return of the network. For excample, as sites start
going in the ground, the distribution of those sites begins to
define the science return from the network. If you know all
of the possible site locations up front (that is they are all
permitted before installation begins), the installations can be
prioritized and the sites built such that if the project needs
to be descope for lack of funds, the last sites that were to be
built were the lowest priority. In reality, it is doubtful that
construction will wait until all of the sites are permitted in
exactly the `right' locations or that all of the sites permits
will be approved in just the right order. Rather it may be that
sites will be built as they are permitted and the permits will
come from as close to the plan as possible.
A compliation of specific comments from committee members:
Text and Figures -
p. 3 Why is PBO adding met. packages to provide "an estimate of
GPS derived tropospheric water vapor over the entire PBO region"?
What science requirement is this responding to?
Where is the science justification for the 100 field receivers?
Why do we need to buy, warehouse, maintain, etc. this equipment?
p.6 oblique views of pie charts are not helpful. Also, it is
not clear to me what the point is of showing how the relative
amounts of the budget are whether or not they are shown as pies.
p. 7 "Based on proven and accepted project management principles."
p. 9 Regional scientist role statement needs to be tightened up.
Why do you need a field engineer reporting to a regional
scientist, who reports to a UNAVCO regional engineer? Fig. 2.3
You should consider not having the Campaign engineer coming on
line until later in the program after some number of stations
have been installed. In fact the after all have been installed,
the campaign engineer could replace the backbone engineer.
Fig 2.3 should be made bigger and show, what I assume to be,
the flatness of the management structure. It looks as if it was
made to fit the formating of the page. Same for Fig. 2.4.
p. 11 The site selection process is controlled by a sensitivity
analysis. Who does the sensitivity analysis? How is that
funded? What is it? How much detail? Is this a full time
person testing hundred's of scenarios per week? How are issues
resolved? Fig. 3.1 needs more arrows indicated the knott of
checks that are being performed. Every box is an approval gate.
The description of the site selection procedure is to simplisitic
given the knowledge of the difficulty of site selection and the
criticalness of it to the overall project success and costs.
p. 12 Who are these `constituents' that the regional scientists
represent? The RS roles and responsibilities and authority need
to be clarified.
Figure 3.1: The basis for the siting plan should primarily be
the reviews from the 2000 Workshop, not the earlier White Paper.
p. 13 The statement "Station permitting and the permit approval
process dominate the time and uncertainty in predicting the rate
of station construction, limits the ability of the stations to be
scheduled for construction, and thus controls the eventual rate of
network deployment" should also include the effect on the costs.
And how is this cost uncertainty budgeted? The stated plan seems
somewhat simplistic and does not seem to incorporate the lessons
learned by SCIGN and others.
p. 14 The bulk permitting process described at the top of the page
has been tried before. Numerous problems exist between local
and angency level control, and the need to pay the agencies for
services such as biological screening, etc. SCIGN's experience
was that permitting can be very slow for large agencies, even
when someone from the agency sits on your coordinating board.
p. 14 Bullet 5, What if the owner wants us to pay a lease? What
then?
p. 14 Should not an attorney for UNAVCO Inc. develop the land
use agreement (LUA)?
p. 15 The plan of the Regional Engineers working to a fixed
schedule published by the Director sounds very optimistic and
inflexible to the dynamic scheduling and environment that may
be likely to exist.
p. 15, Sec 4.1. The regional engineer description reads like
a general contractor. Why is the Regional Engineer acting as
the general contractor? Why isn't the installation contractor
responsible for his own schedule and supplies? Why is UNAVCO
seemingly warehousing everything and micromanageing the
construction? What expertize do they have in that?
It reads like a management structure is being set up where
UNAVCO is too much in the middle of making decisions at a
very low level for its subcontracts. A problem is that the
drillers and the station installation subconstracts, for example,
will be dependent on UNAVCO delivering on schedule a schedule,
equipment, construction permits (not LUAs), Health and Safety
reports, etc. which the subcontractors will have to work
into their schedule with their other clients. This puts the
cost risk on UNAVCO and I don't see the rationale for this.
Also, it requires a much higher level of coordination among
all parties than would be needed if the installation contractor
were responsible. Why not put the reponsinility for coordination
of the contractor's activities on the contractor and let them
propose a schedule that the UNAVCO contract manager (Regional
Engineer) approves?
Are the drilling and installation teams different and do they
work at different times? That could cause a coordination and
scheduling problem.
p. 18 Why 1-2 weeks before operational? Sounds arbitrary and
counter to the well planned executed implementation that is
being suggested. Should it not be 1-2 days? Why not?
Fig. 5.1 Get rid of the yellow gradient background unless it
means something? Add a time line to this figure. Also, on other
figures, the background colors, if they don't add anything to
the content and message, should be removed.
p. 19 What PBO requirements dictate the receiver requirements?
For example, why ethernet? What if it costs too much, can we
live without it? Sounds like a desire and not a requirement.
p. 22 Why is UNAVCO Inc. purchasing the components and shipping
them off to someone else? Doesn't this add costs such as handling
and additional shipping costs? Are you envisioning a cost savings
from no overhead by the subcontractor? Why not write that into the
subcontract rather than take on the logisitical expense of being
a parts distributor?
--------
Budget -
There seemed to be no budget for administrative staff for staff
support. Some dedicated staff will be needed for processing
shipping paper work, site permits, data entry, etc.
The need for level of reserve is not argued in the proposal.
Some reserve is certainly necessary given the high uncertainty
in a number of the tasks and activities. The proper level
of reserve needs to be argued and justified at the WBS level.
My gut feeling is that 13% may not be enough.
There is concern over the credibility of the cost estimate. In
particular, it was somewhat premature to be descoping the science
already on the first draft of the proposal.
A cost comparison table should be developed and perhaps included
for GPS and strainmeter stations based on the known/estimated
costs from existing networks and showing their estimated cost
for PBO. The table should be broken down by distance and type
of station, such as Alaska, urban, remote, etc.
The choice of equipment design, 3-component vs. 1-component,
and the maturity seemed to be unresolved and contributing to
the cost uncertainty. This needs to be resolved.
Processes -
Station distribution maps (Dots Maps) - There needs to be some
process where dots are translated to specific sites. A strategy
needs to be developed regarding the assumptions that can be made
with respect to moving sites around for scientific reasons and/or
cost control.
Having lots of oversight committees passing judgement on whether
or not this site or that site should be built and how should
be avoided. The proposal team should consider developing a
requirements document. This document should be both specific
enough about the mapping of the science goals and priorites to
the observations that it could be used though out the project
life to provide guidance to the field engineers, etc when making
decisions that affect the quality of the science, but should
not be so specific that it is unuseable.
The PBO Director seems to be written in as a micro manager and/or
doer of nearly all tasks. This needs to be clarified.
--------
Figure 3.1: The basis for the siting plan should primarily be the
reviews from the 2000 Workshop, not the earlier White Paper.
Figure 3.3: This flowchart is actually more difficult to follow than
the bulleted description on the next page; I'd omit the figure.
Section 3.4: It is not clear to me that it is a good idea to use the
borehole casings as GPS monuments; and if this adds to the complexity
and time spent finding sites, it may not save much money compared to
building a GPS monument.
On the positive side, I think the plan for permitting makes good
sense, especially hiring some professionals (but is this really part
of what an engieering firm does, or are there more specialized groups?).
Section 4.2: The breakdown into construction phases is very good. I found
the monumentation discussion confusing: a DDB (deep-drilled braced)
monument will be used, except that in bedrock a SDB (shallow-drilled
braced) will be used, and in unconsolidated material a concrete pier:
this seems like a contradictory set of statements. (I also do not think
the concrete pier should be called deep anchored, but that is a separate
technical matter).
For a strainmeter installation, grout is placed into the hole, not
pumped.
It is not completely clear to me that there has in fact been a
community consensus on what should go into boreholes: the most thorough
discussion I am aware of came from the ad-hoc strainmeter group at
the Fall 2001 Earthscope meeting, and I'd hardly call this a community
effort.
Section 4.3, para 1: meteorological not metrological; and what is "this"
in the last sentence?
Section 4.3, para 2: wind power seems like an invitation to high
maintenance.
It is not necessarily less expensive. Construction and maintenance costs
may offset any savings over other alternatives. The windmill should
be collocated away from the antenna, requiring power to be run some
distance in a buried conduit. It will need to be firmly attached to the
ground with sufficient batteries to survive low wind days. In addition
it adds siting requirements that are more restrictive, increasing the
recon costs and influencing the network topology.
Section 5.2, para 2: Full-scale production of GPS systems, or purchase?
Section 5.2, para 3: State that the 100 recivers will be for survey-mode
Section 5.3: Figure 5.2 is purely decorative: omit it.
Section 5.4: The list of communications options seems odd in a couple of
ways. One is that StarBand is not really meant for data communications (as
opposed to consumer Web access), so I am not sure this will work. The other
is that if satellite won't work, it is difficult to picture cell-phone as
a backup.
Though UNVACO has tested StartBand, I believe that this was boot legged
on a consumer account, not an operational, business account. There may
be hidden issues with their support of PBO's activities.
I agree that the choice of cell phone seems odd the way it is presented.
Also, while cell phone rates are low for consumers, businesses pay a
significantly different rate, and the rate structures change frequently.
Hence, the cost of cell phone is not well constrained, but seems costly.
What about CDPD?
Of course, of all the sections, this one is the most ripe for integration
into the overall effort: USArray has to deal with telemetry too.
Section 5.6: Again, it is not clear what type of monument will be
used in which circumstances. I do not think the concrete pier is the
"emerging choice of the GPS research community". (I did mention this
to Mike Jackson and he plans to change it).
Again, the proposal needs to better justify its plans and decisions.
Section 6.2: It is not clear to me how th UNAVCO Facility will pre-test
strainmeters, or that they are the right group to do so. This is the first
of a number of cases in which what is right for GPS is carried over to
a quite different measurement system.
Section 7.2: A positive note: I like the data plan. One implication
is that the Regional centers will not in fact store data onsite for very
long, so they can be quite modest in scope.
Section 7.3, last para: for my taste, this has too much detail about
the software especially relative to the amount of detail presented on
other, more significant elements of the project.
Section 8: As indicated, it is not self-evident that the IGS model will
work cleanly for strain data, at least at first. I don't want to be too
tedious on this, but have to point out some differences between GPS and
strain:
*)Raw GPS data (carrier-beat phase) are very structured: that is,
they result from quite substantial processing of a heavily-engineered source
signal. This makes automated processing easier.
*)GPS data are highly redundant. A day of 30-second data from N sites
produces about 1.e5N values/day; the number of values out of the processing
is more like 500 (orbit and EOP) plus 50N (position and troposphere): about
1000:1 compression. For strain data there is no obvious equivalent compression.
*)And, many fewer man-hours have been spent developing software for
dealing with strain data. For the old-timers, remember hand-editing cycle
slips?
The upshot is that the idea (valid for GPS) that there are standard
processing and products, just waiting to be implemented, is still a goal for
strainmeter data. The proposal seems to assume that the state of play in
the two areas is equivalent.
Section 8.1: Who will decide when specialized datasets should be made
available?
Section 8.2: There is an ambiguity between how often solutions are produced
(daily) and their latency (delay after real time). Latency of 1 day is
useful for spotting problems (but for this the analysis can be rough),
and after the occasional large earthquake (so, once overy couple of years
at most); but on almost all days, it is not scietifically important. If
there is a tradeoff between latency and completeness/quality, I'd go for
the latter: there should at least be (as in global earthquake products)
a range of products produced with differing latencies.
In addition, what science goal/requirement or technical problem is
driving the need for low latency?
Seciton 8.3: It is stated that the Analysis Center Coordinator needs to know:
"The ACC will be a full-time positionn with the incumbent having
a strong geodetic background with a detailed understanding of strainmeter
error correction, GPS analysis strategies, and the application of
reference-frame constraints to GPS solutions."
Perhaps I forgetting someone, but there appears to me to be
exactly one person who could meet these requirements, and John Beavan
seems very happy in New Zealand.
Section 9.2: I'll raise again the question of what priority to give
overseas projects: low, or none? Either will need to be justified.
And the bigger issue of what process is used to decide on the priority
for any project? What will be the demand to use these receivers once
we have 1200 installed around western North America? Who will use them?
Is this the right number of receivers? Will not PI's have their own?
This reminds me of the original operating mode for UNAVCO.
Section 10 (or maybe elsewhere): do we have to promise to advance K-12
education? This does seem unlikely.
Last modified Tuesday, 08-Nov-2005 02:34:51 UTC
