Return to:

Main menu

Steps in preparing returns

File scopes

File structures

Base element definition listing

Derived element definition listing

Glossary

Edit validation rule listing

Using Valpac.Net

 

 

Edition: 20 January 2009

 

D.1        PROCESSES INVOLVED IN PREPARING RETURNS

 

1.1          Overview of the processes

 

For the unit record data collection each year institutions will need to make two (or optionally three) submissions of student data to the Department, one submission of staff data and one submission of Space data. For each of those submissions, each institution will need to undertake the following processes:

 

(a)          Install the Departmental software package VALPAC.Net on the PC to be used to undertake validation of the data in the files and to generate reports from the data. 

 

(b)          Prepare collection data files as flat files in ASCII (plain text) format outside the VALPAC.Net software.   Place the ASCII files in the appropriate subfolder within the \VALPAC\ folder system that the VALPAC.Net installation program will have created.

 

(c)          Import the data from the ASCII files into VALPAC.Net's Access database as "tables" in that database.  In this document, "tables" is used to refer to the Access database tables, not to cross tabulations, which are referred to as "crosstables" or "crosstabulations". The instructions for installing and running VALPAC.Net explain what file names are to be used for each ASCII file and what directory each file needs to be placed in prior to using VALPAC.Net's import functionality.

 

(d)          As an alternative to steps (b) and (c), small institutions may wish to enter data directly into the Access tables within VALPAC.Net, using its inbuilt editor functionality.  At any time VALPAC.Net users can modify data in the Access data tables using the VALPAC.Net editor.

 

(e)          Produce and inspect "validation reports" from the Access tables.

 

(f)           Produce and inspect "distribution reports" from the Access tables.

 

(g)          Produce and inspect the "summary reports" from the Access tables.

 

(h)          If any of the student returns is being prepared, produce from the Access tables and inspect, the funded credit report and the SAPSE-type reports 2.12 and 2.13

.

(i)           If the 2nd or 3rd submission of the student return is being prepared, produce from the Access tables and inspect, the funded credit report, the completed funded credit report and the subsidy credit report.

(j)           Generate an encrypted and compressed file in ZIP format containing the Access database and the validation reports and send this file it to the Department using ftp.

These processes are described briefly in this section. Then Section D2 the specific steps involved in preparing the Student Collection return are explained in detail, and in Section D3 the specific steps involved in preparing the Staff Collection return are explained in detail.

1.2          Preparing ASCII files outside VALPAC.Net

By whatever means are appropriate each institution will need to gather and prepare data according to the specifications, definitions and data rules which are provided in these sections of this document:

Section (E):  Scopes of the ASCII files.

              This section explains what data are to be assembled in what files and dates of submissions required.

Section (F):  Structure of the ASCII files.

              This section explains what data elements (sometimes known as "fields") are to be included in each file and the structure of each file.

              Section (G):  Data Element Dictionary (DED) for the base elements to be included in the ASCII files

              This section is a data element dictionary ("DED") which provides a definition of each data element for which data are to be reported in the ASCII files, including what coding scheme is to be used in reporting of data for the element.  These are referred to as the "base data elements".

             

              In a related "derived elements" DED (See Section H) definitions are provided for "derived" data elements.  Derived data elements are generated from data reported in the base data elements.  They are used in presenting data in summary reports and in the credit reports (e.g. data for age groups). 

              Institutions are NOT required to assemble or report data for the derived elements.  The data for derived elements can be generated using a VALPAC.Net function.  The reason that definitions of the derived elements are provided in this document is so that users of VALPAC.Net will be able to understand how data which appear in summary reports and credit reports have been generated by VALPAC.Net.

Section (I):  Glossary

              This section provides a definition of terms used in the DED for the base data elements (e.g. "Course credit value", "Course level"). 

Section (J):  Determining credit values for courses in universities

              This section provides an explanation of how credit values for courses in universities are to be determined.  For courses in technikons, credit values are to be determined in accordance with SAPSE Report 151.

Section (K):  Edit validation rules

              This section provides an explanation of the data "rules" that are checked by the VALPAC.Net software when it generates a validation report.  Rule violations will be reported by VALPAC.Net in those cases where the specifications in the data element dictionary or file scopes have not been adhered to (e.g. invalid codes have been reported, data in related tables are inconsistent with each other).

1.3          Installing VALPAC.Net

VALPAC.Net needs to be installed on a PC which is going to be used by the institution to import the data from the ASCII files, to validate the data in the files, and to generate reports from the data.  Installation instructions and instructions about how to use VALPAC.Net is provided in this section:

Section L:  Installing and using the Departmental software package "VALPAC.Net"

It is strongly recommended that those instructions be read and understood prior to undertaking any of the following processes.

The installation of VALPAC.Net will lead to the automatic creation of the folder structures that it requires for the student and staff unit record collections.  The folder structure and an explanation of the files that are to appear in each folder is explained in Section L.

1.4          Importing the data from ASCII files into tables in VALPAC.Net's Access database

The ASCII files are then to be placed by the user in the appropriate folder on the user's PC (as indicated in Section L.  The data in the files are then to be imported into tables within VALPAC.Net's Access database.  This is done through the appropriate VALPAC.Net function, as is explained in Section (L).

In the import process, if VALPAC.Net detects an incorrect structure in any of the source records (e.g. the record is too short or too long), it automatically ensures that during the loading process the structure of the data from the source record will amended so that it is suitable for the Access database table.  The user will be informed about the number and types of these amendments at the end of the importing process, and a short message will be included in the Comments field of the amended records in the Access database. 

However, these important points need to be noted:

(a)           During importing, VALPAC.Net cannot actually correct missing or incorrect data.  If records are too short, the image of the record from which VALPAC.Net imports the data will be padded with spaces.  If records are too long, the excess data in the images of the records will simply be excised.  VALPAC.Net will identify errant records.  However, it is the user's responsibility to determine what was wrong with the original data.  If the action taken by VALPAC.Net is inappropriate, the user will need to fix the original data, recreate the source ASCII files, and then re-import the data from those ASCII files into VALPAC.Net.

(b)          VALPAC.Net will NOT correct the structure of records in the original ASCII file.  However, once the data are loaded into the VALPAC.Net tables, if the user wishes to export the amended data in the form of ASCII files containing records that do have the correct structure, there is an exporting function in VALPAC.Net that can be used to do that.  However, as is indicated in the preceding paragraph, the data in the exported files may not be correct.

Data can be re-imported at any time from source ASCII files.  This enables users to import data, then if there are errors found in the data, the data can be corrected outside VALPAC.Net and correct data imported to replace the previously imported data.  The VALPAC.Net import function enables newly imported data to be either appended to the existing data or to be used as a replacement of (all of) the existing data.

If the collection year is 2000 or later for the student collection, or 2001 or later for the staff collection, VALPAC.Net's summary reports and distribution reports will calculate and contrast differences between data for the current collection year and data for the prior year.  For that to occur, the ASCII files for the prior year's collection will need to be imported into VALPAC.Net as tables in the Access database.  That importing can be done in the current collection year, or more likely would have already been done in the process of preparing the returns for the 1999 collection year.

If the collection year is 1999 for the student collection, or 2000 for the staff collection, the summary reports and distribution reports will not calculate and contrast differences from the prior year, because there is no facility in VALPAC.Net for importing data for 1998 into the Access database.

1.5          Editing data in VALPAC.Net tables

At any time the VALPAC.Net function can be used to edit data in the Access data tables or to add records to the tables or to delete records from the tables.  The function is explained in Section L.  However, using VALPAC.Net to enter all the collection data directly into the Access tables would be a tedious process and is not recommended. 

Bad data detected in the Access data tables can be amended by using VALPAC.Net's editor function.  However, if the original source data in the institution's production database are not also amended this will mean that the data submitted to the Department will be different to the data in the institution's records and its production database.  This is a potential source of confusion.  If errors are detected in the data in VALPAC.Net's Access database, a sounder procedure is to amend the original production database, regenerate the ASCII file/s, then to re-import the data into VALPAC.Net.

1.6          Producing and inspecting validation reports

After data have been imported into one or more of the Access tables the data in the table/s are then to be validated.  The appropriate VALPAC.Net function is used to generate "validation reports" that will list any violations of the validation rules which are listed in Section (K).  VALPAC.Net MUST be used to generate the validation reports.

If data are inconsistent with the validation rules they will generate "fatal" or "warning" error messages in the validation report.  All fatal errors MUST be corrected before the institution’s data are sent to the Department.

Some data may generate a warning error.  In such cases institutions are required to check the validity of the data.  Data containing warning errors will be accepted by the Department if the reporting institution has examined the data to ensure it is correct and provides an explanation about warning errors to the Department.  Such an explanation is NOT to be on a case by case basis;  the statement must be in general terms.

The VALPAC.Net function provides two types of reports:  a "detail" report which shows errors listed by individual record, and a "summary" report which shows a summary of the errors found.  Users are able to control the amount of output in the report in these ways:

(a)           By specifying whether the detail report or the summary report or both reports are to be generated.  In case there are many errors in the report, in initial runs of the report, it is advisable to generate only the detail report, which is only one page in length.

(b)          By selecting the edit validation rules for which details are to be shown in the detail report.  This enable the user to have details shown in the report but only for the particular rule or rules of interest.  Irrespective of what rules are selected, information about rule violations for all rules will be shown in the summary report.

All validation reports are displayed on the screen and can be printed then, if required.  All validation reports are saved as tables in the database and can be viewed and printed later, if required.

1.7          Producing and inspecting distribution reports

Although data within a database table may be correct according to all the validation rules, further quality assurance work is required to ensure that the data are reasonable and accurate.  For instance, if all students are coded as "M" indicating that they are all males, this will satisfy the validation rule that checks that all records are coded as either "M" or "F".  However in most institutions the data would be incorrect because it is unlikely that there are no female students in the institution. 

VALPAC.Net's "distribution reports" are very useful in the process of checking that data are reasonable and accurate.  Instructions about how to generate distribution reports are given in Section L.  This type of report comprises counts of all the codes which have been used for a data element in an Access table.  For example a distribution report will show how many records are coded “F” and how many are coded “M” for the gender element.  The report shows such data as counts and percentages for the current year and (if the collection year is 2000 or later for the student collection, or 2001 or later for the staff collection) parallel data for the previous year, and percentage changes between years. 

Distribution reports are also very useful in the process of locating the causes of violations of the validation rules.  The VALPAC.Net function has a facility for selecting a range of values for which data are to be tabulated in the distribution report.  This can be done element by element

The user can select the elements for which tabulations are to be provided in distribution reports.

All distribution reports will be displayed on the screen and can be printed then, if required.  All distribution reports are saved as tables in the database and can be viewed and printed later, if required.

1.8          Producing and inspecting summary reports

The VALPAC.Net "summary reports" are also very useful in the quality assurance process of checking that the data are reasonable and accurate.  There are two summary reports for the Student Collection:  the Student Summary Report and the Course Registrations Summary Report.  There are two summary reports for the Staff Collection:  the Staff Profile Summary Report and the Staff FTE Report.  Instructions on how to generate summary reports are given in Section L.  These reports can be used for two purposes:  inspecting the distribution of values which are reported for each element, and comparing the distribution of values for the current year's data with the distribution for data for the prior year. 

(a)          Distribution of values for data for the current year

              The summary reports will show data from the Access tables in aggregated frequency distribution format.  

              Users should inspect the distributions of data for the current year that are shown in the summary report.  If the distributions of values for the current year do not appear to be reasonable and consistent with other information known to the institution, steps must be taken to fix the data prior to sending them to the Department.

(b)          Comparing data with data for the prior year

              If the collection year is 2000 or later for the student collection, or 2001 or later for the staff collection, the summary reports will show the distribution of values for selected data elements aggregated from the database tables for the current year, and a parallel distribution of values extracted from the tables for the prior year's collection, and the variations from year to year as percentages.

              Changes in the distribution of values for the current year versus the prior year need to be inspected.  Where there are large, unusual or unanticipated changes, this may be a reflection of errors in the data rather than reflecting real changes.  The institution needs to take steps to ensure that the variations are real and understandable, and are not indicative of errors in the data.  If they are indicative of errors in the current year's data, then those errors will need to be fixed.

The summary reports are similar in some respects to the distribution reports.  However, there are a number of differences between the two types of reports.  First, some of the content of the summary reports present grouped values (e.g. age groups) for "derived elements", whereas distribution reports show values for all codes encountered for each "base data element".  Second, the data elements for which data are to appear in distribution reports can be controlled by the user through "filters", whereas the content of summary reports cannot be controlled by the user.

The use of filters allows a subset of cases to be tabulated in a summary report.  For instance the user may be interested in generating a summary report for only instances where a qualification being undertaken has an approved status, an/or the entrance category  is "first-time entering".  For the Student Summary Report, filters that are available relate to these elements:

(a)          Qualification type

(b)          Qualification requirements status

(c)          Qualification approval status

(d)          Entrance category

(e)          Citizen-resident status

(f)           Race

(g)          Gender

(h)          All qualifications or a headcount

For the Course Registrations Summary Report, filters that are available relate to these elements:

(a)          Qualification type

(b)          Qualification requirements status

(c)          Qualification approval status

(d)          Funding status

(e)          Entrance category

(f)           Citizen-resident status

(f)           Race

(g)          Gender

In addition, the Course Registrations Summary Report can be used to tabulate either course credit value or completed course credit value.

Users are able to select the elements that are to be tabulated in summary reports. 

Prior to generating the report, VALPAC.Net checks to ensure if the tables which contain data for derived elements are up to date.  If those tables are not up to date, VALPAC.Net will regenerate them automatically.  If any of the data tables are edited through the VALPAC.Net editor, or if any ASCII file is imported, the derived tables will be determined by VALPAC.Net to be out of date.  Therefore VALPAC.Net will then update the derived data tables prior to generating the summary report.  While the user has an option for forcing the regeneration of the derived data tables, normally this does not need to be done, and updating can be left to be undertaken automatically by VALPAC.Net.

All summary reports are saved as tables in the database and can be viewed and printed later, if required.

1.9          Producing and inspecting the credit reports

The VALPAC.Net credit reports are generated from the student collection data.  There are three types of report:

Report type

When report is to be generated

Sub-reports which can be generated

SAPSE table equivalent

Funded credit

Submission 1, 2 and 3

Data for contact mode

SAPSE table 2.9 minus 2.19

Data for distance mode and mixed mode

SAPSE table 2.19

Completed funded credit

Submission 2 and optional submission 3

Data for contact mode

SAPSE table 2.11 minus 2.21

Data for distance mode and mixed mode

SAPSE table 2.21

Subsidy credit

Submission 2 and optional submission 3

1/2 of values in Funded credit report for Contact mode plus 1/2 of values in Completed funded credit for Contact mode

SAPSE table 2.10 minus 2.20

1/3 of values in Funded credit report for Distance/mixed mode plus 1/3 of values in Completed funded credit report for Distance/mixed mode

SAPSE table 2.20

There is a report generation option in the VALPAC.Net software which enables the reports to be generated including or excluding experiential training.  This option allows the generation of reports equivalent to SAPSE tables 2.17 and 2.22.

All credit reports are saved as Excel-compatible worksheets in files that are separate to the VALPAC.Net database.  These worksheets viewed within VALPAC.Net and printed later, if required.

Instructions about how to generate the credit reports are given in Section L. 

1.10        Producing and inspecting SAPSE-type reports 2.12 and 2.13

VALPAC.Net has functionality that enables reports roughly similar to the SAPSE reports 2.12 and 2.13 to be generated.  These reports contain details about 2nd-order CESM codes for all students and for students who have fulfilled the requirements of a qualification.

The SAPSE reports are saved as Excel-compatible worksheets in files that are separate to the VALPAC.Net database.  These worksheets viewed within VALPAC.Net and printed later, if required.

Instructions about how to generate the credit reports are given in Section L. 

1.11        Generating the encrypted compressed return to be sent to the Department

VALPAC.Net has a function which generates a compressed file in ZIP format, containing the institution's Access database plus the validation reports.  The database and reports are encrypted.  The encryption will be undertaken automatically by VALPAC.Net using a 12-character key to be input by the user.

Prior to encrypting and generating the compressed file, VALPAC.Net will automatically generate all the validation reports.  This ensures that the validation reports sent to the Department are up to date.  Because the generation of the validation reports takes some time, users should not invoke the VALPAC.Net's encrypt/compress option until after they are ready to package the database for despatch to the Department.

Instructions about how to generate the encrypted compressed file are given in Section L.

The compressed file is then to be sent to the Departmental contact person specified in Section C via ftp (File Transfer Protocol).