5i' SECTION 3 CHECKING THE QUALITY OF THE INTERNATIONAL TELEPHONE SERVICE Recommendation E.420 CHECKING THE QUALITY OF THE INTERNATIONAL TELEPHONE | SERVICE - GENERAL CONSIDERATIONS 1 Quality of service parameters An adequate picture of the level of quality of service (QOS) in the network can be defined by a set of parameters which are measured, registered and data processed. In Recommendation E.800 a set of performance concepts is defined in order to provide a satisfactory description of the qual- ity of service, and the interconnection of those concepts is shown. Each performance concept can be impaired by a number of particular causes. These causes, either singly or in groups, lie behind the failure symptoms observed by the user. A user views the provided service from outside the network and his perception can be described in observed quality of service parameters. The link between the observed quality of service param- eters and the impairment causes can be indicated in the form of tables. Five main observed quality of service parameters are derived; they reflect the quality of: i) providing the customer with the ability to use the desired services; ii) furnishing a desired level of service for: - connection establishment, - connection retention, _________________________ Such tables can be found in the handbook cited in [1]. - connection quality, - billing integrity. These main parameters can be supervised by quality of service indicators (e.g. efficiency rate, call cut-off rate, etc.). Objectives can be set for these indicators and can be revised at regular intervals. When a deterioration of these supervision indicators is detected, or when an improvement programme is started, more data must be collected by measurements to permit a more detailed analysis in order to locate the impairment causes which lie behind the observed problem areas. 2 Methods of measuring the quality of service 2.1 The following methods of measuring the quality of service are described: 1) service observations by external means; 2) test call (simulated traffic); 3) customer interviews; 4) internal automatic observations. 2.2 Administrations are recommended to draw up a programme for observations and tests designed for assessment of circuits and equipment, supervision of operators and evaluation of the quality of service given to subscribers. It would be desirable if telephone Administrations were to exchange statistics on quality of service. 2.3 Table 1/E.422 relates to the manual and semi-automatic observations of the quality of international automatic and/or semi-automatic service. It provides in particular a check of the percentage of unsuccessful calls due to technical faults (equipment shortages or failures). Table 2/E.422 relates the same information as Table 1/E.422 but does not include information which can only be obtained by operators listening in (automatic observation). Table 1/E.423 relates to observations on traffic set up by operators. It provides, in manual and semi-automatic service, a means of determining the efficiency of international circuits, of assessing the work of operators and the quality of transmission. Table 2/E.423 summarizes observations of the time-to-answer by operators. The table is compiled by automatic means. Table 1/E.424 is used to record the results of test calls undertaken especially when the observations shown in Table 1/E.422 make it clear that the percentage of faults is too high. The use of customer interviews as a method of measuring tele- phone service quality is the subject of Recommendation E.125 which is particularly concerned with the determination of sources of user difficulty which may arise when making an international automatic telephone call. Recommendation E.426 contains a general guide to the expected percentage of effective international call attempts. Table 1/E.427 may be used to supplement the information con- tained in Table 1/E.422 when the observations shown in that table make it clear that the percentage of faults due to customer diffi- culties is too high or the outcome of the application of Recommendation E.125 demonstrates the need for additional informa- tion. Recommendation E.425 describes the data that might be taken from the switching centres with respect to quality of service, and the exchange of that data. 2.4 Paying attention to the quality of service of the incoming traffic stream is of major importance, since the incoming Adminis- tration is in a better position to improve the situation. In the past less attention has been paid by several Adminis- trations to the quality of service (QOS) on incoming calls than on outgoing calls. This situation should not persist in the future. Therefore, in addition to the measurement of QOS of the outgo- ing traffic stream which is described in this series of Recommenda- tions, Administrations are strongly advised to observe the incoming traffic stream with the aim to improve the QOS. 3 Other sources of information on the quality of service The following sources are useful to consider when trying to improve the quality of service: - subscriber complaints (see Annex B); - other Administrations or organizations such as INTELSAT (SPADE reports); - operators contacting maintenance staff for direct action; - operators giving information on QOS: if operator traffic is significant one might consider organizing the flow of this type of information by establishing "trouble codes", e.g. echo, no tone, no answer, etc.; - reports from "national" switching centres: the QOS as experienced by the subscriber does not only depend on the international network and the network of the country of destination but also on the national network of the country of origin; - user organizations/large companies: as large companies have much to gain from an improved QOS they might be wil- ling to cooperate with Administrations; - holding time versus conversation time measure- ments; - average conversation time; - traffic measurements; - transmission measurements. ANNEX A (to Recommendation E.420) A possible approach to integrate activities measuring the quality of service into an overall problem-investigating process The flowcharts of the resource allocation process and a typi- cal problem identification procedure are given in Figures A-1/E.420 and A-2/E.420. The numbers 1) through 10) in the figures correspond to the processes described below. 1) The exception threshold is set to detect possi- ble isolated destinations. It is up to the individual Administra- tion to set the value. 2) A destination could be regarded as being under isolated condition when the bid frequency is significant enough to show that there is some demand to the destination (e.g. 20 attempts per day) without or nearly without answer. 3) The most practical way to find out whether improvements may be possible is "consultation with other Adminis- trations". 4) Apply, if possible, network management actions, e.g. alternative routing. 5) The destination priority, P , for each destina- tion is calculated as follows: Paid minutes or the revenues can be used. P = BID x (TABR - MABR ) where BID | s the number of total bids to the destination during a certain period of time (for example, 1 month); In case ABR | annot be used, ASR (answer seizure ratio) is con- sidered to be an acceptable substitute. Seizures, TASR and MASR are then applicable. TABR | s the target ABR (answer bid ratio) performance which is expected as the result of the service improvement activi- ties; MABR | s the measured ABR to the destination during the same period with BID . The TABR | s set for each destination and can be based on the average historic ABR and should be higher than that value. In order to comply with Recommendation E.426, S 2.2, the TABR | to be used in the formula for P given above should not be lower than the MABR experienced one period earlier. 6) In order to comply with S 2.4, it is suggested to consider also the total international incoming traffic stream as one of the elements which require QOS improvements. It should be noted that the procedure can be well applied to domestic destina- tions, for example, on an area code basis, and can be applied on an incoming route basis. 7) Perform detailed analysis: when possible, moni- tor circuit group performance and do analyses on a destination code basis. It is essential to be aware of "killer trunks" (though observation of the QOS is not directly intended to discover killer trunks). 8) Discuss possible improvements with counterpart. 9) In Recommendation M.710 (General maintenance organization for the international automatic and semi-automatic service) the basic maintenance elements, their functions and the cooperation between the elements are described. Recommendation M.1230 (Assessment of the performance of the inter- national telephone network) gives guidance on the relationship between service quality observations, network performance assess- ment and maintenance procedures. It should be noted that the QOS very much depends on the proper operation of maintenance elements and maintenance procedures. Therefore, Administrations faced with QOS problems are strongly advised to be attentive to the mainte- nance Recommendations contained in Volume IV. 10) If this procedure does not lead to a success- ful conclusion, then an escalation procedure may be required (see Recommendation M.711). Figure A-1/E.420, p. Figure A-2/E.420, p. ANNEX B (to Recommendation E.420) Utilization of customer complaints to improve the quality of service for international traffic It is possible to use subscriber complaints to control processes if the organization of an Administration provides possi- bilities for centralized collection of these complaints. The collected data can be processed statistically to provide useful indications for the operations and maintenance staff to correct problems and, in so doing, improve the quality of service. Three aspects are relevant in the processing of the data: - the data itself; - the statistical processes; - the analysis of complaints. B.1 Data to be collected The trouble report of an individual customer may be subjective and unqualified since it is usually made by a person, who is not well trained for observation of quality of service. Therefore it will be necessary to make sure that the information about the com- plaint is as reliable as possible and useful for identifying the possible impairment of the network that caused the complaint. Examples (see also the handbook cited in [1]): - data concerning the subscriber numbers involved (route, destination); - data concerning the observations during the unsuccessful call attempt(s), or the disturbed call; - time of the observation by the customer. B.2 Statistical processes to improve the reliability of the data Reliable data is obtained by statistical processing of large numbers of complaints (e.g. an average value during a certain period of time). In order to achieve this, the following methods are considered to be useful: 1) choose complaints whose possible causes seem to relate to impairments of the network; 2) accumulate complaints for a certain period of time, for example, one month or one week, depending on the number of complaints; 3) calculate the ratio of complaints statistically from accumulated data, for example, complaint-to-completion ratio (CTCR), for the chosen period of time: CTCR = umber of effective call attempts ________________________________ x 100% It is practical to use the CTCR in combination with one or more classification aspects (see S B.3) such as "per destination". B.3 Analysis of complaints It is necessary to identify the possible impairment of the network causing the complaint and smoothly clear this impairment in order to actually improve the quality of service. To accomplish this, the complaint needs to be processed into data useful to net- work maintenance organizations in localizing the possible impair- ment. The following methods are considered useful: 1) classifying complaints by category of failure; 2) classifying complaints by destination, route (or circuit group) and/or area code; 3) time of day analysis. This may be effective in identifying impairments that may not be apparent when looked at on a total day basis; 4) highlighting relative changes or trends in the statistical data. These changes are likely to reflect a change of the network status and are useful indications along with the values themselves. For example, a rapid increase in the statistical value (e.g. the ratio of complaints) may reflect a new impairment of the network. Reference [1] CCITT Manual Quality of service, network management and network maintenance , ITU, Geneva, 1984. Recommendation E.421 SERVICE QUALITY OBSERVATIONS ON A STATISTICAL BASIS 1 Definitions 1.1 service observation F: observation de la qualite de service S: observacion de la calidad del servicio Monitoring to obtain a complete or partial assessment of the quality of telephone calls, excluding test calls. 1.2 manual observation F: observation manuelle S: observacion manual Monitoring of telephone calls by an observer without using any automatic data-recording machine. 1.3 automatic observation F: observation automatique S: observacion automatica Monitoring of telephone calls without an observer. 1.4 semi-automatic observation F: observation semi-automatique S: observacion semiautomatica Monitoring of telephone calls using equipment which records some data automatically. For example, equipment in which informa- tion, such as exchange being observed, number dialled by the sub- scriber, metering pulses and time of call, is recorded automati- cally on some means suitable for data processing. The observer merely has to key in a code indicating the condition observed. 2 Relative merits of manual, automatic and semi-automatic observations 2.1 The three methods mentioned above in SS 1.2, 1.3 and 1.4 are not exclusive; for example, automatic observations may be used to supplement observations taken by an operator. It was considered in 1968 that the need for automatic observations would increase in view of the heavy cost associated with manual or semiautomatic observations on the rapidly expanding international network. It was also considered that automatic observations would not entirely supersede observations taken by an observer within the foreseeable future. The relative merits of the three methods can be assessed as follows: 2.2 Manual observation Provides all the data required in Tables 1/E.422 and 1/E.423. Observations can be carried out with a minimum of equipment. Observations can permit the detection of a number of abnormal- ities which cannot be detected automatically, e.g. very poor speech transmission (item 5.2 of Table 1/E.422), or difficulty with audi- ble tones encountered in the international service (item 6.4 of Table 1/E.422). 2.3 Semi-automatic observation Provides all the data required in Tables 1/E.422 and 1/E.423. There is a saving in staffing costs compared with manual observation. Greater accuracy compared with manual observation is possible due to the fact that there is an automatic recording of the number dialled, the time of the call, etc. It is possible for the observer to devote greater attention to the more critical conditions being checked during observations of calls. The results are produced in a form suitable for subsequent mechanized analysis. Owing to the reduction of costs it is possible to obtain a larger sample for the same expenditure. Semi-automatic equipment may be converted, during certain hours of the day, to automatic operation. 2.4 Automatic observation Operating cost is minimum (staff reduction). Continuous observation is possible. It is possible to have a larger sample. Human error is eliminated. Automatic processing of data is facilitated. Conversational privacy is ensured. Control of the time at which observations are made is facili- tated. Some of the differences between internal and external automatic observations are given below: 2.4.1 Internal automatic observations can be made in the switching centre itself, on the incoming side or the outgoing side or in between, according to the way the switching centre is engineered: a) Only line signals, such as seizure, answer, etc. can be monitored, and also register signals as long as they do not pass through the exchange in an end-to-end signalling procedure. b) Signals received are only monitored if the exchange itself operates correctly in that respect. c) Item b) applies also to outgoing signals. If there is a fault in the exchange it can happen that signals have not been sent in the appropriate way without the exchange being aware of it. More information on this type of observation technique is given in Recommendation E.425. 2.4.2 External automatic observations are made by means of monitoring equipment which is supervising the traffic on incoming or outgoing lines: - All signalling signals can be monitored. - The detection of tones, speech and data is possi- ble if advanced equipment is used. - This observation technique provides all the data required in Table 2/E.422 and Table 2/E.423. - The application is very flexible and can be used instead of manual or semi-automatic observation techniques. 3 Time of observations The results of all observations taken over the whole day should be recorded in Table 1/E.422 or Table 2/E.422. In the case where observations are not taken over the whole day the observation period is recorded under the heading "Time of observations" and should include the three busiest hours of the day. 4 Observation access points 4.1 Observations for Table 1/E.422 or Table 2/E.422 should be carried out from access points as close as possible to the outgoing international exchange. The following access points can be considered: For definitions of test access points see Recommendation M.700. See also Recommendation M.110. i) outgoing relay set of an international circuit ("exchange" side), i.e. international circuit access point ; ii) incoming relay set of a national circuit; iii) link circuits of the international exchange Observations will be made only while the call is being set up, and a few seconds after the called subscriber's reply. When the circuit access point is used for observation of international calls it is possible that the service quality of the international exchange may not be checked by either international or national observation programmes. Preferably, and where technically feasible for the most com- plete results, observations for Table 1/E.422 should be carried out as close as possible to the international exchange on the national side. This would be more representative of service to the sub- scriber, and allows observation of call failure at the outgoing international exchange. Where it is not possible to make the dis- tinction between failures in the outgoing international exchange, and failures beyond this exchange, or where there is a meaningful advantage in doing so, observations should be taken on the outgoing side. It is necessary to state in Table 1/E.422 or Table 2/E.422 the access point where the observations have been made, as observations obtained at each one of the three access points mentioned above are not comparable. 4.2 Observations for Table 1/E.423 must be carried out from access points on the operators' positions. 5 Number of observations 5.1 Service observing programmes should be established in such a manner that statistical results obtained be as reliable as prac- ticable bearing in mind the cost of obtaining large samples. 5.2 According to the studies carried out by the CCITT in 1964-1968, the quantities shown below are considered the minimum quantities to provide a general indication of the quality of ser- vice 5.2.1 Table 1/E.422 The minimum number of observations per outgoing circuit group for Table 1/E.422 should be 200 per month when more than 20 cir- cuits are included in a group, 200 per quarter when there are between 10 and 20 circuits in a group and 200 per year if there are less than 10 circuits in a group. 5.2.2 Table 1/E.423 The minimum number of observations for Table 1/E.423 should be 200 per quarter when there are more than 20 circuits in the group, 200 per semester when there are between 10 and 20 circuits and 200 per year when there are less than 10 circuits in the group. 5.2.3 Transit traffic Where an outgoing circuit group also carries transit traffic it is desirable to obtain data for each destination country reached via this circuit group. In principle, the number of observations for each destination should be obtained as indicated above. To accomplish this, one should use for each destination country its corresponding number of erlangs and derive from these erlangs a theoretical number of circuits. However, where only a very small amount of traffic is handled, e.g. less than 5 erlangs, each Administration may wish either to make a smaller number of observations or (e.g. in case of no com- plaints) no observations at all and rely on the information obtained at the transit exchange. 5.3 The number of observations specified above will provide a general indication of results on quality of service in certain broad categories. Administrations may desire more accurate results especially for the individual categories in Table 1/E.422. Attention is drawn to Table 1/E.421 which gives the number of observations required to obtain a certain degree of accuracy. H.T. [T1.421] TABLE 1/E.421 _________________________________________________________________________ { { _ | 5% _ | 0% _ | 5% _ | 0% _ | 5% _ | 0% _________________________________________________________________________ 2 3136 2178 1600 1225 1030 880 4 1536 1067 784 600 500 440 6 1003 696 512 392 330 290 8 736 511 376 288 245 215 10 576 400 294 225 195 170 12 469 326 239 183 150 132 14 393 273 201 154 128 112 16 336 233 171 131 112 98 18 292 202 149 114 95 80 20 256 178 131 100 85 70 30 149 104 76 60 50 42 40 96 67 50 38 30 24 50 64 44 33 25 20 16 _________________________________________________________________________ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | TABLE 1/E.421 [T1.421], p. Annex to Table 1/E.421 Examples of use of Table 1/E.421 Example 1 - It is estimated from previous results that a par- ticular type of failure occurs on about 4% of calls. If it is required to confirm, with 95% confidence, that the existing failure rate is between 3% and 5% (i.e. _ | 5% of 4%), then observations must be made on a random sample of 1536 calls. Example 2 - For an expected failure rate of 2%, observations must be made on a random sample of about 1200 calls (1225 in the table) to predict, with 95% confidence, that the true percentage is between 1.2% and 2.8% (i.e. _ | 0% of 2%). This means that when 200 observations are taken over a period it is necessary to take the "rolling average" of conditions over six periods. The rate of failure for a number of categories important from the maintenance point of view is expected to be about 2%. Example 3 - After observations have been taken and the rate of failure in the sample has been calculated, the table may be used in a "backward" direction to give a rough indication of the accu- racy of the result. Suppose that out of a sample of 1000 observations, there were 29 failures due to cause "X" and 15 failures due to cause "Y". The rates of failure in the sample due to X and Y, respectively, are then 2.9% and 1.5%. From the table, it is apparent from this sam- ple of 1000 calls that the true rate of failure due to X has an accuracy of about _ | 5% (i.e. is between 1.9% and 3.9%), and that due to Y has an accuracy of about _ | 0% (i.e. is between 0.8% and 2.3%). 6 Exchange and analysis of the results of observations 6.1 Exchange of the results of observations The following periodicities are proposed for the exchange of results between Administrations: Table 1/E.422 or Table 2/E.422 - a monthly exchange is desir- able; Table 1/E.423 or Table 2/E.423 - a quarterly exchange is desirable. Nevertheless, in the case of small groups of circuits (less than 20 circuits) the information should be exchanged after 200 observations have been made but never later than one year in any case; attention is drawn to the fact that less than 200 observations are of little value. Results of observations will be reported without delay: - to the Administrations and the network analysis point of the country where observations are carried out; - to the Administrations and the network analysis point of the other country (including transit Administrations and their network analysis point when involved). The benefits to be derived from service observations tend to decrease if there is any increase in the time taken to make the results available to those who can take action to bring about an improvement. The results of service observations according to Tables 1/E.422 and 1/E.423 should therefore be made available to the Administration in the countries of destination as soon as pos- sible after completion of the observation period and in any case within six weeks. 6.2 Analysis of observation results An analysis of the results should be carried out in the coun- try of origin as well as in the country of destination. Some Administrations have found it useful to distribute to other Administrations concerned, service observation statistics in the form of graphs. Recommendation E.422 OBSERVATIONS ON INTERNATIONAL OUTGOING TELEPHONE CALLS FOR QUALITY OF SERVICE 1 Objectives concerning Table 1/E.422 and Table 2/E.422 1.1 The purpose of service observation in the international service is to assess the quality of service obtained by the calling subscriber. Consequently, it is essential to have factual or objective recording of observations (i.e., successful and unsuc- cessful calls), and to present them in the form of a table (see Table 1/E.422 for manual or semi-automatic observations and Table 2/E.422 for automatic observations). 2 Manual or semi-automatic observations (Table 1/E.422) 2.1 Table 1/E.422 should be capable of being completed through the use of a wide range of observation facilities, i.e. from the simple to the sophisticated. H.T. [1T1.422] TABLE 1/E.422 Observations of international outgoing telephone calls for quality of service _____________________________________________________________________________ Country of origin | Point of access: { Outgoing international exchange | } National side | Group of circuits | Link circuits | { Service | | ?02 | utomatic | ua) Service | | | emi-automatic | ua) } Outgoing side | Period: from | to | Time of observations | _____________________________________________________________________________ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ________________________________________________________________________________________________________________ Number Percentage Category Subtotal Total Subtotal Total ________________________________________________________________________________________________________________ { 1. Calls successfully put through (see note 1) } ... ... ________________________________________________________________________________________________________________ { 2. Ring tone received but no answer } ... ... ________________________________________________________________________________________________________________ { 3. Unsuccessful calls: Positive indication of congestion, including subscriber busy, from beyond the outgoing international exchange. Visual signal, tone or recorded announcement } ... ... { 3.1 Subscriber busy/congestion indicated by visual signal } ... ... { 3.2 Subscriber busy/congestion indicated by busy/congestion tone } ... ... { 3.3 Congestion indicated by a recorded announcement } ... ... ________________________________________________________________________________________________________________ { 4. Unsuccessful calls: Other visual signals, tones or recorded announcements, not positively identified as category 3 or 8 } ... ... 4.1 Visual signal received ... ... 4.2 Tone received ... ... { 4.3 Recorded announcement received } ... ... ________________________________________________________________________________________________________________ { 5. Unsuccessful calls for other technical reasons } ... ... 5.1 Wrong number obtained ... ... { 5.2 Abandoned due to very poor speech transmission } ... ... { 5.3 No tone, no answer after waiting ... seconds } ... ... { 5.4 Reception of answer signal when the called party does not reply } ... ... { 5.5 Other failures of a technical kind } ... ... ________________________________________________________________________________________________________________ { 6. Unsuccessful calls due to incorrect handling by the calling party } ... ... 6.1 Wrong number dialled ... ... 6.2 Incomplete number ... ... { 6.3 Call prematurely abandoned before receipt of signal, tone or announcement (within less than ... seconds) } ... ... { 6.4 Call prematurely abandoned after receipt of ring tone (within less than 30 seconds) } ... ... { 6.5 Other failures due to incorrect handling } ... ... ________________________________________________________________________________________________________________ { 7. Total calls monitored | categories 1-6) } ... 100 ________________________________________________________________________________________________________________ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Table 1/E.422 [1T1.422], p. H.T. [2T1.422] TABLE 1/E.422 (cont.) __________________________________________________________________________________________________ Number Percentage Category Subtotal Total Subtotal| Total __________________________________________________________________________________________________ { 8. Unsuccessful calls: Positive indication of failure from outgoing international exchange } ... { 8.1 Congestion on outgoing international circuits } ... 8.2 All other indications ... __________________________________________________________________________________________________ { 9. Successful calls with defects. These calls are included in category 1 } ... { 9.1 Non-reception of answer signal on chargeable calls } ... { 9.2 Call with impaired intelligibility but not abandoned } ... { 9.3 Other calls with defects but not abandoned } { ... } __________________________________________________________________________________________________ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | a) Delete whatever is inapplicable. Note 1 - A successful call is one that reaches the wanted number and allows conversation to proceed. All successful calls are entered in category 1. However, a successful call may or may not have noticeable defects. Successful calls with noticeable defects should also be entered in category 9. Note 2 - With the exception noted above for categories 1 and 9, the results of one call observation should be entered under one category only, namely the most appropriate one from 1 to 6. Note 3 - Administrations should periodically exchange necessary information to interpret the observation data recorded under categories 4.1, 4.2 and 4.3. Tableau 1/E.422 [2T1.422] p. 5 2.2 Specialized training of observers should be kept to a minimum. 2.3 The table should be self-explanatory so that reference to detailed how-to-complete instructions is unnecessary. 2.4 The major categories should be selected such that: - they identify the major factors adversely affect- ing the quality of service; - they are suitable for the centralized processing of observation results. 2.5 To permit the orderly collection of data for human factors studies to identify sources of difficulty in customer use of the international (automatic) telephone service, Recommendation E.427 contains an additional table to Table 1/E.422. 3 Comments concerning the use of Table 1/E.422 3.1 Table 1/E.422 summarizes observations made on outgoing automatic and/or semi-automatic traffic , on a country of origin to a country of destination basis. A separate form should be used for each country of destination, and if required, for each group of circuits to which traffic to a country of destination has access at the outgoing international exchange (or exchanges). It is not necessary to make observations on both automatic and semi-automatic services. An Administration may select the service to be observed, provided that the service is the majority of the traffic to the country of destination. 3.2 For an explanation of the point of access, see Recommendation E.421, S 4.1. 3.3 The result of each call observed should be entered only under the most appropriate category. In the case of several faults on one call, the most significant cause of failure should be entered. 3.4 In completing Table 1/E.422 reference should be made to the following explanations. 4 How to fill in Table 1/E.422 Category 1 - To ensure objective recording and to avoid producing a biased sample resulting from the exclusion of calls which require subjective assessment, the successful call is defined as a call that reaches the wanted number and allows conversation to proceed. All non-abandoned calls are entered into category 1 and of these calls those which are subjectively adjudged to be defective are also entered into category 9. Thus it is required of the observer to make two entries for successful calls with noticeable defects. Enter in category 1 then, calls successfully put through. This includes answered calls for which a clearback signal is received after some words have been spoken, without knowing for what reason the call is abandoned. If it is observed that the caller has dialled a wrong number, the call will be entered under 6.1. Category 1 will also include calls put through correctly to opera- tor positions, information services, or to machines replying in place of the subscriber or to their equivalents. Category 2 - Enter in this category calls on which ring tone was heard but the subscriber did not answer before the attempt was abandoned, the caller having waited at least 30 seconds after com- mencement of ring tone before clearing forward. (See category 6.4 if the call was abandoned less than 30 seconds after ring tone com- menced.) Category 3 - Enter in this category all unsuccessful calls in which a positive indication of subscriber busy or congestion beyond the outgoing international exchange had been encountered, either by visual signal, tone or recorded announcement. Congestion encoun- tered on common control equipment should be entered in this category as well (e.g. no "proceed-to-send" signal). Where a posi- tive indication of these conditions has not been received, enter in category 4. Categories 3.1, 3.2 and 3.3 are entered for the specific indi- cation received. When more than one indication is received, e.g. visual signal and audible tone, only one entry should be made. In this case, the preferred order of entry should be tone, announcement, visual sig- nal. Category 4 - Enter in this category all other indications on unsuccessful calls whether by visual signal, tone or recorded announcement that cannot be positively identified and entered in category 3 or 8. Categories 4.1, 4.2 and 4.3 are entered for the specific indi- cation received. When more than one indication is received, e.g. visual signal and audible tone, only one entry should be made. In this case, the preferred order of entry should be tone, announcement, visual sig- nal. Category 5 - Enter in this category those calls which fail for technical reasons not included in categories 3, 4 and 8. Category 5 subdivides as follows: Category 5.1 - Calls on which the wrong number was obtained, although the caller dialled correctly. Category 5.2 - Calls abandoned by the caller because of very poor speech transmission, although the answer signal was received. (See category 9.2 if speech transmission is poor but the call is not abandoned.) In some countries observers may be required to cease listening immediately after conversation is established, thus reducing the number of calls that would be reported in this category. Category 5.3 - Calls on which the dialling information was correctly and completely sent, but the caller received no signal, tone or announcement before abandoning the call, having waited for at least the specified period before clearing forward. The value of this time period left open under this category should be filled in by the Administrations of the ori- ginating country according to its experience in this matter. The prescribed value may differ depending on the international destina- tion. It is, however, recommended to limit the number of such dif- ferent quoted periods to a maximum of three values (e.g. 10, 20 or 30 seconds or any other value considered pertinent by the Adminis- trations concerned). Category 5.4 - Calls on which an answer signal was received, although the called subscriber did not answer. Category 5.5 - Call failures due to technical reasons which are unable to be entered in categories 5.1 to 5.4. These should be very few, if any, and this category is provided in case they do arise. All possible information about these failures should be supplied as an attachment to the summary of the table. This category includes calls abandoned due to reception of a clear-back signal while connecting with the extension number (PBX). Category 6 - Enter in this category all unsuccessful calls which have failed due to incorrect handling by the caller (sub- scriber or operator). Category 6 subdivides as follows: Category 6.1 - Calls on which it was determined that the number which should have been dialled was different from the number actually dialled. Category 6.2 - Calls on which it was determined that the number dialled had insufficient digits to be successful. Category 6.3 - Calls on which the digital information was correctly and completely sent, but the caller abandoned the call without receiving any signal, tone or announcement, and without waiting for at least the specified period. The value of the time period left open under this category should be filled in by the Administrations of the originating coun- try according to its experience in this matter. The prescribed value may differ depending on the international destination. It is, however, recommended to limit the number of such different quoted periods to a maximum of three values (e.g. 10, 20 or 30 seconds or any other value considered pertinent by the Administration con- cerned). The value quoted under category 6 must be the same as that quoted under category 5. Category 6.4 - Calls prematurely abandoned after receipt of the ringing tone on which the caller disconnected less than 30 seconds after the ringing tone commenced. (See category 2 if the call was abandoned after more than 30 seconds had elapsed from the time of commencement of ringing tone.) Category 6.5 - Calls which failed due to incorrect han- dling by the caller which cannot be classifed under categories 6.1 to 6.4. All possible information about these failures should be supplied as an attachment to the summary of the table. As in categories 5.5, these should be very few, if any. Category 7 - Enter in category 7 the number of calls moni- tored (categories 1-6). Category 8 - Category 8 will be useful for those Administra- tions which observe on the national side of the outgoing interna- tional exchange. (See Recommendation E.421, S 4.1.) Positive indi- cations of failure, congestion or other, are to be entered here. They are not to be included with categories 1-6, which give the data for calls monitored for category 7. Thus, when category 8 is viewed with categories 3 and 4 a more complete picture is provided of quality of service received by the caller. Category 9 - Entries in category 9 are for successful calls (entered in category 1) which encountered defects, but which were not abandoned. They are thus automatically included in the total of category 7. Category 9.1 - Enter here chargeable calls for which no answer signal was received. If abandonment should be detected on such calls, enter in category 5.5. Category 9.2 - Enter here calls on which poor speech transmission was observed, but the call was not abandoned. (See category 5.2 if the call was abandoned.) All possible information about these calls should be supplied as an attachment to the sum- mary of the table. Note that in some countries observers may be required to cease listening immediately after conversation was established, thus reducing the number of calls that would be reported under this category. Category 9.3 - Enter here calls encountering switching, signalling or transmission defects, but which were not abandoned and which cannot be classified under categories 9.1 or 9.2. 5 Automatic observations (Table 2/E.422) Considering the limitation of abilities of automatic observa- tion equipment (for example, automatic observation equipment cannot understand announcements) and the variety of signals used in sig- nalling systems, the table recommended for CCITT Signalling System No. 5 is given below. H.T. [T2.422] Tableau maintenu - sans correction = MONTAGE H.T. [T1.423] Tableau maintenu - sans correction = MONTAGE H.T. [T2.423] TABLE 2/E.423 Automatic observations of the time-to-answer by operators Interna- tional outgoing exchange Circuit group Service: semi-automatic Table 2/E.422 [T2.422], p. 6 Comments concerning the use of Table 2/E.422 6.1 Table 2/E.422 summarizes observations made on outgoing automatic and semi-automatic traffic, on a country of origin to a country of destination basis. A separate form should be used for each country of destination, and if required, for each group of circuits to which traffic to the country of destination has access at the outgoing international exchange (or exchanges). 6.2 For an explanation of the point of access, see Recommendation E.421, S 4.1. 6.3 The result of each call observed should be entered only under the most appropriate category. In the case of several faults on one call, the most significant cause of failure should be entered. 6.4 As the function of sound analysis by automatic observation equipment is not concerned with the signalling system used and since some signalling systems e.g. Signalling System No. 6 have more information exchanged in the signalling system than those of sound signals, it is expected that the proposed table will be applied to all signalling systems for the present. 6.5 In completing Table 2/E.422 reference should be made to the following explanations. 7 How to fill in Table 2/E.422 Category 1 - The successful call is defined as a call that allows conversation to begin between subscribers, or allows to begin sending facsimile or data. This includes calls put through to operator positions, information services, or to machines replying in place of the subscriber or to their equivalents. In other words, the successful call is such that the automatic observation equip- ment detected voice on both sending and receiving lines, or that it detected sending tone of facsimiles or data, or that it detected voice on the receiving line after receipt of answer signal. Category 2 - This category includes those calls for which the automatic observation equipment detected ringing tone , but there was no answer signal and the clear-forward signal was sent 30 seconds after the detection of ringing tone. Category 3 - Enter in category 3 all unsuccessful calls for which a positive indication of subscriber busy or congestion beyond the outgoing international exchange has been encountered, either by visual signal (busy-flash signal) or by tone (also includes no "proceed-to-send" signal). Category 4 - Enter in category 4 unsuccessful calls for which the automatic observation equipment detected a tone, but could not classify it, or the equipment detected announcement (that is, it detected voice on receiving line without answer signal). Category 5 - Enter in category 5 those calls which failed for technical reasons not included in categories 3, 4 and 8. Category 5 subdivides as follows: Category 5.1 - Calls on which the dialling information was completely sent, but the automatic observation equipment received no signal, tone or announcement and it received a clear-forward signal after a specified period. The value of this time period left open under this category should be filled in by the Adminis- trations of the originating country according to its experience in this matter. The prescribed value may differ depending on the international destination. It is, however, recommended to limit the number of such different quoted periods to a maximum of three values (e.g. 10, 20 or 30 seconds or any other value considered pertinent by the Administrations concerned). Category 5.2 - Calls on which an answer signal was received, although the called subscriber did not answer. In other words, calls for which the automatic observation equipment received an answer signal, although it detected no voice on receiving line. Category 5.3 - Failed calls due to technical reasons which are unable to be entered in categories 5.1 and 5.2. For example, a call for which there was a busy-flash signal after receiving ring- ing tone. Category 6 - Enter in category 6 all unsuccessful calls which have failed due to incorrect handling by the caller (subscriber or operator). Category 6 subdivides as follows: Category 6.1 - Calls on which the dialling information was completely sent, but the automatic observation equipment received no signal, tone or announcement and it received a clear-forward signal within a specified period. (For this period, see category 5.1 above.) Category 6.2 - Calls prematurely abandoned after receipt of the ringing tone on which a clear-forward signal was received less than 30 seconds after the ringing tone was detected. Category 6.3 - Calls which failed due to incorrect han- dling by the caller which cannot be classified under categories 6.1 and 6.2. For example, a call for which the automatic observation equipment received an answer signal after receiving ringing tone, and then the ringing tone stopped, but the equipment could not detect any voice either on the sending line or the receiving line. Category 7 - Enter in category 7 the number of calls moni- tored (categories 1-6). Category 8 - Category 8 will be useful for those Administra- tions which observe on the national side of the outgoing interna- tional exchange. Positive indications of failure, congestion or other, are to be entered here. Category 9 - Entries in category 9 are for successful calls (entered in category 1) which encountered defects. Category 9 sub- divides as follows: Category 9.1 - Calls on which no answer signal was received, but the conversation was begun. Category 9.2 - Calls which encountered switching or sig- nalling defects, but on which the conversation was begun. Recommendation E.423 OBSERVATIONS ON TRAFFIC SET UP BY OPERATORS 1 Comments concerning the use of Table 1/E.423 1.1 This table summarizes observations relating to manual and semi-automatic outgoing traffic originated by operators. These observations will be made, if possible, during the whole call dura- tion service, if there is no problem regarding the efficiency of international circuits 1.2 Administrations should, if possible, make a distinction between the different types of call, e.g. station-to-station, per- sonal and collect calls ; they should use a separate column for each under the heading "Type of call". 1.3 For collect calls, the times to be recorded will be those observed in the country where the call request was made. 1.4 It is recommended that these observations be spread over the whole day. 1.5 Each outgoing Administration will select the international circuit groups on which observations should be carried out. 1.6 In completing this table, reference should be made to the following explanations: 2 How to fill in Table 1/E.423 (Traffic observations determined by the operators) Category 1 - This category should show the mean duration of calls observed which are successful and have been charged for ("effective" calls). Category 2 - This category will show the mean chargeable duration of all effective calls observed. Category 3 - This category will show, for each type of observed call, the average time per effective call during which the international circuit has been occupied for manoeuvres or for call preparation. This average should be based on the time during which the international circuit is held: a) to obtain information concerning the called number; b) to obtain information about routing and trunk codes; c) to call operators, in the incoming international exchange; d) to exchange information on how to set up the call; e) to (or attempt to) obtain the called number even when it is engaged or does not reply; f ) to (or attempt to) obtain the called person (in personal calls); g) between replacement of the receiver by the called person and release of the circuit; h) because the operator is holding the circuit (whether she is on the line or not) and for any other reasons for which the circuit is engaged. H.T. [T1.423] Tableau maintenu - sans correction = MONTAGE H.T. [T2.423] TABLE 2/E.423 Automatic observations of the time-to-answer by operators Interna- tional outgoing exchange Circuit group Service: semi-automatic Table 1/E.423 [T1.423], p. The times listed above, which exclude the conversation time, should be added together. This total should be divided by the number of effective calls observed during the period in question to obtain the value to be entered in Table 1/E.423. Category 4 - The number of effective calls observed con- sidered in category 1. Category 5 - The mean number of times the international cir- cuit was seized per effective call (see category 3). This number is usually obtained by meter recordings. Category 6 - The mean number of attempts (as specifically defined hereafter from the operating point of view) to set up a call. Should the operator try several times to set up a call while continuously occupied on that call, all these operations must be considered as being one attempt. Similarly, if the operator makes several tries to set up a call and each time encounters a conges- tion or busy condition and if, after the last try, she informs the caller, only one attempt must be entered. Calls to information ser- vices or to obtain routing particulars, and all calls not directly related to the establishment of a call or to information required by the caller, should not be considered as attempts and should not be included. The total number of attempts during the period of observation should be divided by the number of effective calls observed in the same period to obtain the mean number of attempts per call. The total number of attempts is usually determined from mark- ings or notations on call tickets Category 7 - The data for this category will be taken from all tickets prepared for the relation concerned, during the period of observation or a comparable period. Category 8 - The mean waiting time for outgoing operators to receive an answer will be indicated in seconds. This average will include both answered and unanswered calls. An outgoing operator waits on the circuit (waiting time) for the period: a) until the incoming operator answers, or b) until she abandons the attempt, should the incoming operator not answer. Thus while mean waiting time relates to the outgoing operator it is also a measure of the performance of the incoming operators. Category 9 - It will be difficult to obtain absolutely com- parable results from all observers for this category. However, the observer should consider the quality of transmission from the sub- scribers' viewpoint, taking into account comments made in this respect by subscribers and the number of requests for conversation to be repeated. Category 10 - This category should include any comments likely to explain the probable cause of difficulties frequently noted during the observations. 3 Automatic observations of the time-to-answerby operators (Comments concerning the use of Table 2/E.423) 3.1 This table summarizes observation of the time-to-answer by operators. 3.2 Administrations should make a distinction between the dif- ferent types of incoming operators if the types of operators are distinguished by the selecting digits. 3.3 It is recommended that these observations be spread over the whole day. 3.4 Each outgoing Administration will select the international circuit groups on which observations should be carried out. 3.5 The time-to-answer of the assistance operator cannot be measured automatically. 3.6 In completing this table, reference should be made to the explanations in S 4. 4 How to fill in Table 2/E.423 (Automatic observations of the time-to-answer by operators) The mean waiting time for outgoing operators to receive an answer will be indicated in seconds. This average will include both answered and unanswered calls. The mean waiting time is defined as the time interval between the instant the outgoing circuit is seized (the seizing signal is sent) and: a) the instant the incoming operator answers, or b) the instant the outgoing operator abandons the attempt (a clear-forward signal is sent). H.T. [T2.423] TABLE 2/E.423 Automatic observations of the time-to-answer by operators International outgoing exchange Circuit group Service: semi-automatic lw(66p) | lw(12p) | lw(30p) | lw(12p) | lw(12p) | lw(12p) | lw(12p) | lw(12p) | lw(12p) | lw(12p) | lw(12p) | lw(12p) | lw(12p) . Table 2/E.423 [T2.423], p. Recommendation E.424 TEST CALLS 1 General Test calls carried out manually or automatically to assess the functioning of international circuits of connections are of four types: a) Type 1 test call A test call conducted between two directly connected inter- national centres to verify that the transmission and signalling on an international circuit of a given group are satisfactory. b) Type 2 test call A test call conducted between two international centres not directly connected to verify transit operational facilities of an intermediate international centre. c) Type 3 test call A test call from an international centre to a subscriber type number in the national network of the distant country, gen- erally as a result of a particular kind of fault. d) Subscriber-to-subscriber type test call A subscriber-to-subscriber type test call is a test call from a test equipment having the characteristics of an average sub- scriber line in one national network to a similar equipment in the national network of a distant country. Test calls types 1, 2, 3 and subscriber-to-subscriber test calls must not interfere with customer traffic. If, however, test calls contributing a significant load on a part of a network are to be made, prior advice should be given to the other Administration(s) concerned. Types 1 and 2 test calls for preven- tive maintenance should be conducted during light load periods. Types 1 and 2 test calls should be conducted as and when required for the investigation and clearance of faults. Type 3 test calls should be conducted only after adequate testing has been done by means of type 1 or 2 test calls and after the distant Administration has made the necessary check in its national network. Type 3 test calls should be conducted during light load periods. In order to find faults in last-choice equipment, circuit mul- tiplication equipment or in-circuit multiplexing equipment, it may be necessary for tests to be carried out at the time when the traffic load approaches the full capacity of the route under test. The agreement of the distant network analysis point will be neces- sary before this test is carried out. Subscriber-to-subscriber type test calls can be made by agree- ment of the network analysis point in the countries concerned. Normally, unless there is a specific agreement between the Administrations concerned, subscriber-to-subscriber type test calls would be considered for fault location after: 1) verifying that there are no evident faults in the international switching centres involved that would cause the poor quality of service or subscriber complaint being investigated; 2) verifying that type 1 or type 2 test calls have _________________________ Recommendation M.1235 describes the use of automatic-to-subscriber test calls in more detail. been made on the international circuits that might have been involved; 3) verifying that there are no evident faults in the national network from the outgoing exchange to the interna- tional centre in the originating country; 4) verifying that there are no evident faults in the national network in the distant country, from the international centre to the called exchange. When test calls are undertaken from the international centre to a subscriber number to verify that there are no evident faults in the national network, such calls should be routed through the international centre on the same path as a normal incoming interna- tional call. Using the test access facilities in the international centre could route calls via a different path thereby masking a fault. When subscriber-to-subscriber type test calls are made, the network analysis point in the two countries should consider such factors as: i) the expected nature of the fault; ii) international accounting agreements; iii) the need for making the test calls in the busy hour; iv) the possibility of causing or aggravating congestion at the time the calls are made. The responding equipments used for subscriber-to-subscriber type test calls could be those used for maintenance of the national network. 2 Results of test calls (see Table 1/E.424) H.T. [T1.424] TABLE 1/E.424 Results of test calls _____________________________________________________ { International outgoing exchange: } { Type of test call Type 1 | ua) } Circuit group: { Type 2 | ua) Type 3 | ua) } { Service | | ?02 | emi-automatic | ua) Service | | | utomatic | ua) } Sub-to-Sub | ua) Period from to _____________________________________________________ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | _________________________________________________________________________________ Number Percentage Category Subtotal Total Subtotal Total _________________________________________________________________________________ 1. Satisfactory tests . | | . | | _________________________________________________________________________________ { 2. Signalling and charging faults } . | | . | | 2.1 Wrong number . | | . | | 2.2 No tone, no answer . | | . | | { 2.3 Absence of a backward line signal } . | | . | | 2.4 Other faults . | | . | | _________________________________________________________________________________ 3. Transmission faults . | | . | | 3.1 Conversation impossible . | | . | | { 3.2 Call overamplified or underamplified } . | | . | | 3.3 Noise . | | . | | 3.4 Fading . | | . | | 3.5 Crosstalk . | | . | | _________________________________________________________________________________ 4. Congestion . | | . | | _________________________________________________________________________________ 5. Other faults . | | . | | . | | . | | . | | . | | _________________________________________________________________________________ Tests carried out . | | 100 _________________________________________________________________________________ | | | | | | | | | | | | | | | | | | | | | | | | Table 1/E.424 [T1.424], p. Blanc Recommendation E.425 INTERNAL AUTOMATIC OBSERVATIONS 1 Definitions 1.1 essential information (of internal automatic observa- tions) The answer seizure ratio (ASR) (see S 1.3) or answer bid ratio (ABR) (see S 1.4), whichever is appropriate in terms of attempts, completed attempts and percentage completed. 1.2 supplementary information (of internal automatic obser- vations) Information on signalling faults, subscriber behaviour and the network. 1.3 answer seizure ratio (ASR) ASR gives the relationship between the number of seizures that result in an answer signal and the total number of seizures. This is a direct measure of the effectiveness of the service being offered and is usually expressed as a percentage as follows: _________________________ This Recommendation also applies in case external moni- toring equipment is used when a route is monitored con- stantly for all or a large (statistical significance) number of calls. Refer to Recommendation E.421, S 2.4. ASR = otal seizures __________________________________ x 100 Measurement of ASR may be made on a route or on a destination code basis. 1.4 answer bid ratio (ABR) Gives the relationship between the number of bids that result in an answer signal and the total number of bids. ABR = otal bids ______________________________ x 100 ABR is expressed as a percentage and is a direct measure of the effectiveness of traffic from the point of measurement. It is similar to ASR except that it includes bids that do not result in a seizure. 2 Merits of internal automatic observations The advantage of internal monitoring is that a large volume of records can be collected. The large volume of data obtained from an internal observation system allows day-to-day evaluation of network performance and, coupled with a good maintenance response, is instrumental in providing the best possible quality of service. The disadvantage is that this method does not have the capability of detecting tones or speech and therefore cannot present a complete representation of all call dispositions. To overcome this disadvantage Administrations are advised to use Recommendation E.422 as well to supplement the data obtained from internal automatic observations. 3 Time of observations The results of all observations taken over the whole day should be recorded. 4 Exchange of the results of observations _________________________ Using these techniques one can improve the quality of service even when no distinction can be made between ring no answer, subscriber busy (or congestion indicat- ed by congestion tone) and recorded announcement. 4.1 The essential information should be exchanged monthly (preferably by facsimile or telex) to all network analyses points of those Administrations who are interested (the analyses points can then make comparisons between different streams going to the same destination). If information on ASR or ABR can be supplied separately for direct routes and indirect routes via transit coun- tries, this should also be exchanged as being essential informa- tion, including the name of the transit country involved. 4.2 With respect to supplementary data such as: signalling faults, failures due to calling subscriber, failures due to called subscriber and failures due to the network, a quarterly exchange of information is appropriate. Because different formats will be required, mail seems the most likely means to be used for exchang- ing supplementary data. 4.3 Besides the monthly and quarterly exchange of information, a direct contact on all aspects should be made (by telephone) as soon as action is required to prevent a persistent drop in the quality of service. 5 Classes of calls The distinction between classes of calls (such as operator-operator, subscriber-subscriber and operator-subscriber) is considered useful in identifying problems relating to the qual- ity of service. This can only be done if the language digit and some of the subsequent digits are analyzed. 6 Destination analysis from service observation data Consideration should be given to include the dialled digits, as observed by the monitoring equipment, in the exchange of infor- mation, especially for the sake of destination analyses (see Recommendation E.420, Annex A). 7 Details about supplementary information for CCITT Signal- ling System No. 5 _________________________ The Administration supplying the data must indicate whether the ASR or ABR is used. The language or discrimination digit is inserted au- tomatically, or by the operator, between the country code (Recommendation E.161) and the national (signifi- cant) number. 7.1 Signalling faults - faulty signals; - time outs, the main item in this category being no proceed-to-send signal; - busy flash. (Since busy flash is applied in many situations, including failures due to calling and called subscriber and the network, it is considered useful to distinguish between busy flash received within 0-15 seconds, 15-30 seconds and after 30 seconds when making destination analysis.) 7.2 Ineffective calls associated with the calling sub- scriber Premature release , to distinguish between release before or after having received ringing tone ; equipment which can detect audible signals is required. 7.3 Ineffective calls associated with the called subscriber Ringing tone no answer cannot be detected without equipment which can detect audible signals 7.4 Network Here only the busy flash can be detected without equipment which can detect audible signals. 8 Equipment impact 8.1 Administrations are recommended to consider inclusion of appropriate facilities in existing and new exchanges to record all or some of the following phases: a) Calls switched to speech position , then: 1) answered; 2) unanswered, but released by calling party; 3) timed out awaiting answer; 4) a call failure signal (busy flash or equivalent) received; 5) timed out after clearback signal ; 6) faulty signal received after answer. b) Calls failing to switch to speech position: 1) clear forward signal received; 2) insufficient digits received; 3) congestion on international circuits; 4) faulty signals received into exchange; 5) signalling fault into next exchange; 6) time out while signalling to next exchange; 7) congestion signal received from next exchange; 8) vacant number received; 9) busy subscriber signal received; 10) line out of order signal received; 11) transferred subscriber signal received. As a minimum requirement one should be capable of determining the answer seizure ratio (ASR) or the answer bid ratio (ABR) if they contain some more information than the information already required for international accounting. 8.2 Another way to assemble date on the quality of service (QOS) on outgoing circuit groups is through event counters. Five event counters already give a reasonable amount of information, three of them being common to the Signalling Systems No. 5, No. 6 and R2: seizure, answer and busy signals Signalling System No. 5 The number of: - seizing signals sent; _________________________ In case the event counting is used to analyze the qual- ity of service to a particular destination, the count- ing should be done separately for each signalling sys- tem. - end-of-pulsing (ST) signals sent; - proceed-to-send signals received; - busy flash signals received; - answer signals received. Signalling System No. 6 The number of: - initial address messages (IAM) sent; - congestion (switching-equipment; circuit groups; national network) signals, call-failure signals and confusion sig- nals received; - address-complete (subscriber-free, charge; subscriber-free, no charge; subscriber-free, coinbox; charge; no charge; coinbox) signals received; - subscriber busy signals received; - answer (charge; no charge) signals received. Signalling System R2 The number of: - seizing signals sent; - congestion [national network (A4 or B4); interna- tional exchange (A15)] signals received; - address complete (charge; subscriber's line free, charge; subscriber's line free, no charge) signals received; - subscriber line busy signals received; - answer signals received. Recommendation E.426 GENERAL GUIDE TO THE PERCENTAGE OF EFFECTIVE | fR ATTEMPTS WHICH SHOULD BE OBSERVED FOR INTERNATIONAL TELEPHONE CALLS 1 General considerations 1.1 The success of call attempts is fundamental to an automatic international telephone service of high quality. 1.2 The periodic observation of completion ratio and the categorization of failures to destination countries together with the exchange of such information between countries are valuable to establish and/or maintain a high service quality. 1.3 The call completion ratio of the national network of a given country, as manifested through its international switching centre(s), affects the efficiency of operation of all countries routing traffic to that country. 1.4 Call completion ratio information can be provided either internally in an SPC international switching centre or externally at the level of the outgoing international circuits in any interna- tional switching centre in which access to the circuits is provided for the purpose of establishing the disposition of call attempts. 1.5 The availability, flexibility and capacity of minicomput- ers provides an economically attractive method of obtaining call completion ratio information with extreme accuracy. This includes the observation of tones when suitable interfaces with the minicom- puter are provided. 2 A guide to the proportion of effective call attempts 2.1 A general guide for the expected percentage of effective call attempts during the mean busy hour and its two immediately adjacent hours, as observed at the originating international switching exchange, is indicated below. An effective call attempt is defined, for this purpose, as one for which an answer is received at the originating international exchange. Faults caused by the originating international exchange shall be excluded to the extent feasible. All attempts which succeed in seizing an interna- tional circuit shall be included in the results: a) low level of effective call attempts: less than 30%; b) medium level of effective call attempts: 30% to 60%; c) high level of effective call attempts: more than 60%. _________________________ See Recommendation E.600. 2.2 When an originating country notes a downward change in the level of effective call attempts towards any destination, the ori- ginating, destination or transit Administrations should initiate investigations to determine and alleviate the underlying causes (e.g. network provisioning, subscriber behaviour). The objective of this action is to avoid degradation in the level of effective call attempts. Recommendation E.427 COLLECTION AND STATISTICAL ANALYSIS OF SPECIAL QUALITY OF SERVICE | fR OBSERVATION DATA FOR MEASUREMENTS OF CUSTOMER DIFFICULTIES IN THE INTERNATIONAL AUTOMATIC SERVICE This Recommendation is provided to permit the orderly collec- tion of data required for special studies to identify sources of difficulty in customer use of the international automatic telephone service. When calls are made to points outside a customer's home coun- try, many different sets of ringing and busy tones are encountered. In order to measure the effect of unusual sounding ringing tones and busy tones on customer behaviour, it has been decided to col- lect data on how long customers listen to such foreign tones as well as to their national tones in order to compare them. The data are to be collected in the same manner as those required for the completion of Table 1/E.422. These data are an extension of those collected for Table 1/E.422, and, as an aid to subsequent analysis, a copy of the current version of that table should be used with the table of this Recommendation. Table 1/E.427 contains questions numbered 1-9. Their relation- ship to the questions of Table 1/E.422 is shown in parentheses. A preferred set of analyses for identifying the statistical significance of differences between data collected from subscribers when setting up national calls and the corresponding data collected from subscribers when setting up international calls is given below. 1 Determine the percentage change in any measure by use of the formula: Change (C i ) = | | fIN jfR _________- fIN HfR _________| | x 100 j = A, B, C i = 0-2, 2-5 . | | , > 30 where fi\dj is the observed frequency of calls of category i in the country j , Nj is the total number of observations in the country j sample, fi\dH is the observed frequency of calls of category i in the home country H , and NH is the total number of observations in the home country sample. 2 Compare the central location of the distributions by use of the Kruskal-Wallis One-Way Analysis of Variance [1]. 3 Compare the "forms" or "shapes" of the distribution by means of the chi-square test [2]. 4 Compare changes in single valued variables, e.g. percentage incomplete-trunk-code, by use the chi-square test. Blanc H.T. [1T1.427] Tableau maintenu - sans correction = MONTAGE H.T. [2T1.427] Tableau maintenu - sans correction = MONTAGE TABLEAU 1/E.427 [1T1.427], p. 10 TABLEAU 1/E.427 (suite) 2T1.427 p. 11 References [1] MARASCUILO (L. | .), McSWEENEY (M.): Non-Parametric and Distribution-Free Methods for the Social Sciences, Wadsworth Pub- lishing Co. , California, 1977. [2] SIEGEL (S.): Non-Parametric Statistics for the Behavioural Sciences, McGraw Hill , New York, 1956. Recommendation E.428 CONNECTION RETENTION 1 Introduction Connection retention is one of the parameters influencing the quality of service (QOS) after the call has been set up. As the assessment of this parameter is difficult and costly, it is recom- mended to investigate only after information from sources such as operator trouble reports, subscriber complaints, interviews and/or service observations indicate that there is a problem. 2 Cut-off call ratio The cut-off call ratio is the percentage of the established calls that are released for a reason other than intentional by any of the parties involved in the call. The cut-off call ratio can only be measured by placing test calls (see Recommendation G.181). 3 Investigations required Before measuring the cut-off call ratio the source information leading to the investigation should be utilized to the extent pos- sible (see Annex B to Recommendation E.420) and the outcome should be compared with other relevant sources. For example, subscriber complaints can be followed up by investigating operator trouble reports. These investigations should lead to a suspect part of the net- work. 4 Exchange of information In case the suspect part of the network is outside the terri- tory of an Administration, the Administration which has responsi- bility for the suspect part of the network should be contacted and informed of the results of the investigations. In the information given, the type of cut-offs should be classified by causes such as "absence of answer signal", "artificial clearback signal", etc. 5 Further investigation required Within the suspect part of the network, the Administration should look for obvious causes of cut-offs such as exchange or facility failures. If obvious causes cannot be identified the Administration should consider test calls. 6 Test calls One or both Administrations may decide to set up a test call programme. International standardized test call types are contained in Recommendation E.424. As stated in that Recommendation, before applying type 3 test calls or subscriber-to-subscriber type test calls, it should be verified that there are no evident faults in the national network. This verification can easily be undertaken by applying non-standardized test calls, for example from the interna- tional centre to a subscriber number in the national network of the same country (see Recommendation E.424, 4) of S 1). The usefulness of standardization in these kinds of test calls is under study in several Administrations. For making test calls on connection retention, the responding equipment used should send an answer signal after 10 seconds fol- lowed by a continuous tone in conformity with Recommendation O.61. The threshold level of the detector in the directing equipment should also be in conformity with Recommendation O.61. 7 Objectives The long-term objectives as well as the allocations are given in Recommendation G.181. In the intermediate time the following requirements should apply: the cut-off call ratio for subscriber-to-subscriber tests should, measured over 24-hour periods, be below 0.5% for 5 minute calls. In any time-consistent hour the call cut-off rate may not exceed 3%. It should be noted that, in general, cut-off problems have a more severe impact on customers' perception of service on routes with a low answer seizure ratio than on routes with a low comple- tion rate. Blanc