FY 1999 Annual Performance Plan

Validation and Verification

HUD needs to achieve better system integration. Too often, the data used by the Department has been suspect or our systems inadequate to generate the proper level of reporting and information. In part, these problems are addressed through the Financial Systems Integration project. This project is discussed in detail in the Means and Strategies section. Also, the Department is making significant progress by strengthening, improving and integrating many of its program systems.

Following are analyses of the individual program areas.

Community Planning and Development

The Integrated Disbursement and Information System (IDIS) consolidates planning and reporting processes across four CPD formula grant programs: Community Development Block Grant (CDBG), HOME Investment Partnerships (HOME), Emergency Shelter Grants (ESG), and Housing Opportunities for Persons with AIDS (HOPWA). The grantees, State and local governments, use an integrated system for consolidated planning, project description, disbursement, and accomplishment reporting. Reporting occurs as accomplishments are attained, reducing the burden of end-of-year reports.

When an activity or project is complete, grantees update the database and report on accomplishments achieved for that activity. Annually, the grantee will also electronically report on all accomplishments to date for each activity or project not yet complete. With a goal of paperless reporting, IDIS permits HUD and the community to produce reports of on-going projects, activities, accomplishments, and expenditures tied to those projects - at any time. These reports constitute a significant part of the CPD annual performance report requirements on activities, and will become valuable tools in local decision-making processes.

In conjunction with IDIS, the Grants Management Process (GMP) System unites CPD's core activities, including grantee consultation, timely processing of grant awards, oversight of grantee compliance with regulatory and statutory requirements, performance evaluation, and technical assistance.

Testing of Baseline Data Sources

A number of reporting sources currently exist in the IDIS and GMP that will be analyzed to verify and validate grantee performance. Within IDIS, the following reporting functions document grantee activity and performance using CDBG, ESG, HOME, and HOPWA funds: Summary of Activities Report, the Summary of Accomplishments Report, Summary of Community Development Accomplishments (CDBG Expenditures by Priority Needs Category), and the Consolidated Plan Annual Evaluation Report (comparing accomplishments to benchmarks established in the Consolidated Plan). As part of GMP, grantee performance is self-evaluated by the Consolidated Annual Performance and Evaluation Report (CAPER). Field staff assess grantee performance throughout the program year, documenting strengths and weaknesses in the GMP system. Such ongoing reviews culminate in the Annual Community Assessment (ACA) and Program Year Letter, officially documenting the Department's conclusions regarding a grantee's performance (in conformance with Title I of the National Affordable Housing Act and the Housing and Community Development Act of 1974).

Testing these data will include ongoing reviews of IDIS and GMP. IDIS data will be sampled to determine accuracy, completeness, and timeliness. Reviews of actual performance data for reporting periods will include comparing projected targets for the program year against actual accomplishment data; comparing performance of other jurisdictions; benchmarking performance against superior performers; and , ensuring the data agrees with data from other sources (including evaluation modules within the GMP System). Analyses of program performance information shall show the extent to which grantees are meeting statutory and regulatory objectives of the programs (including costs, services delivered and beneficiaries) and performance data requirements as specified by IDIS.

Public and Indian Housing

As outlined in their FY 1997 Quality Assurance Plan, PIH's monitoring capabilities are enhanced through the use of the "Performance Measure Indicators" reports module of the office's Integrated Business System (IBS). In addition, the Multifamily Tenant Characteristics System provides valuable information to PIH and other entities through the extraction of data from the system. Through the use of both systems, PIH manager's are able to monitor the status of performance measures.

PIH has not experienced any problems with the current systems; however, because new performance measure goals have been established, it will be necessary to modify some of the existing databases. In addition, PIH is in the process of seeking additional funding to develop systems to support the new FY99 performance measure goals established to implement the Native American Housing Assistance and Self-Determination Act (NAHASDA).

As part of the Quality Assurance Plan to implement the verification/validation process, PIH has included in its FY 1999 budget submission funding to utilize the Performance Measure Indefinite Quantity Contracts procured by the Office of Policy Development and Research.

Public Housing Management Assessment Program (PHMAP)

The Department is developing a revised performance evaluation system that will expand and improve on the current PHMAP by evaluating public housing authorities' (PHAs) performance in four areas: physical conditions, financial stability, management, and customer service. The new system will answer previous criticisms of PHMAP by placing greater emphasis on: physical conditions of properties, objective evidence and third party verification, basic real estate functions, and input from those served directly by the system (the "customers"). The result of this effort should be the increased confidence of everyone concerned -- HUD, PHAs, residents, Congress, the public -- that it accurately determines whether PHAs are doing an acceptable or outstanding job and are providing acceptable basic housing conditions. This is critical because the consequences of that determination are substantial -- less scrutiny and additional flexibility for PHAs to be determined performing well, and intensive technical assistance and short time deadlines to improve or avoid losing management control for PHAs to be determined troubled. The new system should better indicate to everyone the degree of success of the public housing program under current funding levels.

This revised PHMAP is currently being developed with the benefit of consultation with public housing authority representatives, representatives of public housing resident groups and others such as housing advocacy groups and local government representatives. The new PHMAP system will be issued as a proposed rule and, like any other proposed rule, subjected to the public comment process. While the rulemaking process proceeds, the new aspects of the system (notably, the independent physical inspection and customer survey) will be refined and tested. Exact implementation schedules will depend upon progress on these tasks, and the final proposal will take into account both that further refinement and testing and the further public comments and consultations.

Section 8 Management Assessment Program (SEMAP)

The Department is currently developing SEMAP in order to objectively measure public housing agency (HA) performance in key Section 8 tenant­based assistance program areas. SEMAP will enable HUD to ensure program integrity and accountability by identifying HA management capabilities and deficiencies and by improving risk assessment to effectively target monitoring and program assistance. HAs could also use SEMAP performance analysis to assess their own program operations.

The Department is developing a final rule based on comments received on its proposed rule. This final rule is expected to be issued during FY 1998. It will commence in FY 1999 and baseline data for performance measurement will be available in FY 2000.

The use of SEMAP will enable the Department to improve its risk assessment and to target effectively monitoring and program assistance to housing agencies (HAs), in key Section 8 tenant­based assistance program areas, that need the most improvement and that pose the greatest risk.

It is proposed that SEMAP will have 15 indicators to assess the HAs performance; the annual HA SEMAP certification and HUD review process; HUD scoring procedures and procedures for designating high, standard and troubled performers; and corrective action plans and strategies for improving performance.

SEMAP will enable HUD to ensure program integrity and accountability by identifying HA management capabilities and deficiencies and by improving risk assessment to effectively target monitoring and program assistance. HAs could use the SEMAP performance analysis to assess their own program operations.

Fair Housing and Equal Opportunity

FHEO utilizes several data sources, including HMDA, GSE, MTCS, RACS, TEAPOTS, IDIS and others. For those measures listed below where current data is not available, development of the new system will be part of their system integration plan. All new systems will be in line with the required systems development procedures and in compliance with the Department's system integration plans and policies. Their FY 1997 Quality Assurance Plan is available on request.


Housing has analyzed their data sources according to its major program activities.

Single Family

    Asset Management [Source of Data: F60 - SFMNS (In-Place)]

    This system provides support for production, rehabilitation and financing of low-cost housing to homebuyers through sales of HUD-owned mortgages.

    Production [Source of Data: F17 - CHUMS (In-Place)]

    In terms of processing, the objective is to have an accurate reflection of HUD's customer service commitment to mortgagees through:

    1. Work Flow
    2. Teamwork
    3. Coordination of contracting resources

    The system, which includes a validation process, is relied upon for tracking of the process. It automatically tracks and reports numbers processed and forms the basis for reporting. Field processing is being consolidated into 4 processing centers in FY98.

    For First Time Homebuyers, the system identifies endorsement of first time homebuyers. System automatically tracks and reports numbers processed. CHUMS report forms basis for reporting. Validation process occurs. Field processing is being consolidated into 4 processing centers in FY98.

    The CHUMS system also identifies endorsement of 203(k) loans. Manual adjustment is made for investor loans and the system automatically tracks and reports numbers processed. Validation process occurs.

    Real Estate Owned (REO) - Sales Performance [Source of Data: A80 - SAMS (In-Place)]

    SAMS is updated and generates reports identifying categories of properties sold for reporting in Commissioner's monthly executive reports. Current validations by staff is considered adequate. Executive report cross-checks data reported to accounting against sales data reported by field.

    Local Goals, including 203(k) Production, Energy Efficient Mortgages (EEMs), REO Inventories, Post Technical Reviews (PTR), Insurance Production (endorsements), and Appraisal Activity. [Sources of Data: F17 - CHUMS and A80 - SAMS (In-Place)

    Lenders/Field Offices enter data into CHUMS, which passes the data to IACS System. Reports are generated weekly, monthly and quarterly and delivered to program offices, operations, and field offices. Reconciliation occurs between the servicing mortgagees and FNMA. Field Office has responsibility of validating data integrity. CHUMS produces reports to the field and headquarters. The data integrity validations are handled by the field. Field office and lender enter data into CHUMS. Weekly, monthly and quarterly reports are generated from CHUMS and distributed to field and headquarters.

Multifamily (MF)

    Production -- Endorsements & Closings [Source of Data: F47 - MIS (In-Place)]

    MF Branch maintains PC-based system which is updated daily. System has edits that kick out any invalid data. No endorsement processing occurs without the official receipt. Validation process occurs through interaction among Field, OGC, Accounting Control Division and headquarters MF program offices. Weekly and Annual reconciliations occur between FOMNS & F47. 202/811 processing is handled using the HUD-290 procedures. Initial data entry in PAS by field. Field offices send in the 290s to MF Headquarters program office for reporting & validating monthly loan closings. OFA provides 202 reports to field and 811 reports only upon request.

    Action Required: Reconciliation is needed between OFA reporting and Program office to ensure data integrity.

    Asset Management [Source of Data: F49 - MARS (In-Place)]

    This system provides support for problems in existing MF portfolio through sale of mortgage notes.

    Resolution of problems in the MF portfolio and expand housing available to low-income renters. [Existing Source of Data: Manual System (in place); Planned Source of Data: RAMS (To be determined during FY98)]

    Originally, MF had planned to use the MF Data Warehouse to identify and track progress of troubled projects. However, there have been some data problems and the data warehouse cannot be used at this time. Overall, MF has been enhancing the quality of their data for quarterly reporting, is looking for ways to better manage the MF data, and is manually tracking the process and progress being made on this goal. Field offices report accomplishments directly to headquarters program office monthly. MF tentatively plans to use the RAMS system for reporting data and for validation efforts. See FY98 Quality Assurance Plan.

    Property Disposition [Source of Data: F46 - PMS (In-Place)]

    The HUD-1 is prepared by closing attorney. MIAS receives HUD-1 confirmation. Field Office enters data into PMS. Closing is not recognized until MIAS receives the HUD-1. System reports can be retrieved by headquarters and field. Data integrity is verified by the PMS contractor and MIAS.

    Resolution of data problems in the existing MF portfolio through validation [Sources of Data: TRACS, FOMNS, PAS (In Place)]

    MF pulls extracts from TRACS for each field office. Geo-coding takes place. Project addresses and identifiers not geo-coded are returned to the field for verification. This process can occur as many times as needed to reach 100% accuracy. Validation occurs directly with the field office and is done on a weekly basis.

    Neighborhood Network Centers [Source of Data: PC Based System Maintained by Aspen Systems (In-Place)]

    Validation occurs and existing process is adequate for reporting purposes.


Content Archived: November 29, 2011