PRECIS support forum http://forum.precisrcm.com/ |
|
Error Code : 539 Message : REPLANCA: http://forum.precisrcm.com/viewtopic.php?f=1&t=70 |
Page 1 of 1 |
Author: | RuthCM [ Tue Jul 28, 2015 3:16 pm ] |
Post subject: | Error Code : 539 Message : REPLANCA: |
Hi, I am running an experiment, using ERA-Interim as boundary conditions, and everything was going perfectly , but then (10 years later) it stopped and I cannot restart the simulation (rerun to complete time). I am guessing is something with the LBC but I cannot find the error. cheers, Ruth Here is part of the leave file ( i cannot attached it so i copied it here) ***************************************************************** Job started at : lun jul 27 11:49:06 CDT 2015 Run started from UMUI Running from control files in /home/rcerezo/umui_runs/aabab-208093502 ***************************************************************** Version 4.5 template, Unified Model , Non-Operational Created by UMUI version 4.5.1 ***************************************************************** jobrunning file: /home/rcerezo/.jobrunning.aabab.9055 Contents: # PIDs for PRECIS experiment aabab 9055 /home/rcerezo/umui_runs/aabab-208093502/SCRIPT semarnat1 This job is running on machine UAS1S0P000A160U, using UM directory /home/rcerezo/precis/um, and test directory . -------------------------------------------------------------------------------- PRECIS version 1.9.2 RUNNING SCRIPT WITH ARGUMENTS: /home/rcerezo/precis/um/vn4.5/utils/restart aabab lun jul 27 09:35:02 CDT 2015 -------------------------------------------------------------------------------- PRECIS version: 1.9.2 Max number of concurrent PRECIS jobs: 2 TYPE is CRUN in /home/rcerezo/umui_jobs/aabab/SUBMIT FROM_PRECIS=true Starting with command: /home/rcerezo/precis/um/vn4.5/utils/umsubmit aabab >>/home/rcerezo/precis/tmp/aabab.restart 2>&1 Your output will be sent to file: /home/rcerezo/precis/precis_out/aabab000.aabab.d15208.t093502.leave warning: commands will be executed using /bin/sh job 154 at Mon Jul 27 09:35:00 2015 -------------------------------------------------------------------------------- Contents of the SAVE_FILE: # These are the local variables which define the experiment # Please note that the region directory must exist job_name: semarnat1 job_desc: test 1 para semarnat runid: aabab region: CORDEX-CA_EXT rcm: HadRM3P source: erain idump: eraina.dai8c10 eyear: 20 emonth: 0 eday: 0 sulphur: Y d360: N umui_base: xacga climate_mean: 30,3,4 cyear: 1960 cmonth: 12 cday: 1 diag_time: b diag_lev: 850,700,500,250,50 out_format: PP name: CORDEX-CA_EXT dx: .44 lonlc: 303.6 latlc: 35.65 ew: 226 ns: 155 plon: 113.98 plat: 75.74 nmpp: 16 moses2: Y -------------------------------------------------------------------------------- Restarted from rerun. Details: -------------------------------------------------------------------------------- PRECIS version 1.9.2 RUNNING SCRIPT WITH ARGUMENTS: /home/rcerezo/precis/um/vn4.5/utils/rerun aabab k0510 18,0,0,0,0,0 CRUN lun jul 27 09:35:02 CDT 2015 -------------------------------------------------------------------------------- Permanent history file : /home/rcerezo/precis/precis_expt/aabab/aabab.phist Possible restart times TYPE Restart dumps ---------------------------- ---- -------------------------------------------- k0610 = 2000/ 6/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.dak0610 k05t0 = 2000/ 5/29 0:00:00z CRUN k0550 = 2000/ 5/ 5 0:00:00z NRUN /home/rcerezo/precis/precis_expt/aabab/aababa.dak0550 k0510 = 2000/ 5/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.dak0510 k0410 = 2000/ 4/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.dak0410 k0310 = 2000/ 3/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.dak0310 k0210 = 2000/ 2/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.dak0210 k0110 = 2000/ 1/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.dak0110 j9c10 = 1999/12/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj9c10 j9b10 = 1999/11/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj9b10 j9a10 = 1999/10/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj9a10 j9910 = 1999/ 9/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj9910 j9810 = 1999/ 8/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj9810 j9710 = 1999/ 7/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj9710 j9610 = 1999/ 6/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj9610 j9510 = 1999/ 5/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj9510 j9410 = 1999/ 4/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj9410 j9310 = 1999/ 3/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj9310 j9210 = 1999/ 2/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj9210 j9110 = 1999/ 1/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj9110 j8c10 = 1998/12/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj8c10 j8b10 = 1998/11/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj8b10 j8a10 = 1998/10/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj8a10 j8910 = 1998/ 9/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj8910 j8810 = 1998/ 8/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj8810 j8710 = 1998/ 7/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj8710 j8610 = 1998/ 6/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj8610 j8510 = 1998/ 5/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj8510 j8410 = 1998/ 4/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj8410 j8310 = 1998/ 3/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj8310 j8210 = 1998/ 2/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj8210 j8110 = 1998/ 1/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj8110 j7c10 = 1997/12/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj7c10 j7b10 = 1997/11/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj7b10 j7a10 = 1997/10/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj7a10 j7910 = 1997/ 9/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj7910 j7810 = 1997/ 8/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj7810 j7710 = 1997/ 7/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj7710 j7610 = 1997/ 6/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj7610 j7510 = 1997/ 5/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj7510 j7410 = 1997/ 4/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj7410 j7310 = 1997/ 3/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj7310 j7210 = 1997/ 2/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj7210 j7110 = 1997/ 1/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj7110 j6c10 = 1996/12/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj6c10 j6b10 = 1996/11/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj6b10 j6a10 = 1996/10/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj6a10 j6910 = 1996/ 9/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj6910 j6810 = 1996/ 8/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj6810 j6710 = 1996/ 7/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj6710 j6610 = 1996/ 6/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj6610 j6510 = 1996/ 5/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj6510 j6410 = 1996/ 4/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj6410 j6310 = 1996/ 3/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj6310 j6210 = 1996/ 2/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj6210 j6110 = 1996/ 1/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj6110 j5c10 = 1995/12/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj5c10 j5b10 = 1995/11/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj5b10 j5a10 = 1995/10/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj5a10 j5910 = 1995/ 9/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj5910 j5810 = 1995/ 8/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj5810 j5710 = 1995/ 7/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj5710 j5610 = 1995/ 6/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj5610 j5510 = 1995/ 5/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj5510 j5410 = 1995/ 4/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj5410 j5310 = 1995/ 3/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj5310 j5210 = 1995/ 2/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj5210 j5110 = 1995/ 1/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj5110 j4c10 = 1994/12/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj4c10 j4b10 = 1994/11/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj4b10 j4a10 = 1994/10/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj4a10 j4910 = 1994/ 9/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj4910 j4810 = 1994/ 8/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj4810 j4710 = 1994/ 7/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj4710 j4610 = 1994/ 6/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj4610 j4510 = 1994/ 5/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj4510 j4410 = 1994/ 4/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj4410 j4310 = 1994/ 3/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj4310 j4210 = 1994/ 2/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj4210 j4110 = 1994/ 1/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj4110 j3c10 = 1993/12/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj3c10 j3b10 = 1993/11/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj3b10 j3a10 = 1993/10/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj3a10 j3910 = 1993/ 9/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj3910 j3810 = 1993/ 8/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj3810 j3710 = 1993/ 7/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj3710 j3610 = 1993/ 6/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj3610 j3510 = 1993/ 5/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj3510 j3410 = 1993/ 4/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj3410 j3310 = 1993/ 3/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj3310 j3210 = 1993/ 2/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj3210 j3110 = 1993/ 1/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj3110 j2c10 = 1992/12/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj2c10 j2b10 = 1992/11/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj2b10 j2a10 = 1992/10/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj2a10 j2910 = 1992/ 9/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj2910 j2810 = 1992/ 8/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj2810 j2710 = 1992/ 7/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj2710 j2610 = 1992/ 6/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj2610 j2510 = 1992/ 5/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj2510 j2410 = 1992/ 4/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj2410 j2310 = 1992/ 3/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj2310 j2210 = 1992/ 2/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj2210 j2110 = 1992/ 1/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj2110 j1c10 = 1991/12/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj1c10 j1b10 = 1991/11/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj1b10 j1a10 = 1991/10/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj1a10 j1910 = 1991/ 9/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj1910 j1810 = 1991/ 8/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj1810 j1710 = 1991/ 7/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj1710 j1610 = 1991/ 6/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj1610 j1510 = 1991/ 5/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj1510 j1410 = 1991/ 4/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj1410 j1310 = 1991/ 3/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj1310 j1210 = 1991/ 2/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj1210 j1110 = 1991/ 1/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj1110 j0c10 = 1990/12/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj0c10 j0b10 = 1990/11/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj0b10 j0a10 = 1990/10/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj0a10 j0910 = 1990/ 9/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj0910 j0810 = 1990/ 8/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj0810 j0710 = 1990/ 7/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj0710 j0610 = 1990/ 6/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj0610 j0510 = 1990/ 5/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj0510 j0410 = 1990/ 4/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj0410 j0310 = 1990/ 3/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj0310 j0210 = 1990/ 2/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj0210 j0110 = 1990/ 1/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.daj0110 i9c10 = 1989/12/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.dai9c10 i9b10 = 1989/11/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.dai9b10 i9a10 = 1989/10/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.dai9a10 i9910 = 1989/ 9/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.dai9910 i9810 = 1989/ 8/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.dai9810 i9710 = 1989/ 7/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.dai9710 i9610 = 1989/ 6/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.dai9610 i9510 = 1989/ 5/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.dai9510 i9410 = 1989/ 4/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.dai9410 i9310 = 1989/ 3/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.dai9310 i9210 = 1989/ 2/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.dai9210 i9110 = 1989/ 1/ 1 0:00:00z CRUN /home/rcerezo/precis/archive/aabab/dumps/aababa.dai9110 ---------------------------- ---- -------------------------------------------- NRUN = Model may be restarted from this time only as an NRUN CRUN = Model may be restarted from this time as an NRUN or a CRUN Directories searched for restart dumps: /home/rcerezo/precis/precis_expt/aabab /home/rcerezo/precis/archive/aabab/dumps /home/rcerezo/precis/archive/aabab Copied /home/rcerezo/precis/archive/aabab/dumps/aababa.dak0510 to /home/rcerezo/precis/precis_expt/aabab Restart dump exists : /home/rcerezo/precis/precis_expt/aabab/aababa.dak0510 Requested restart time : k0510 = 2000/ 5/ 1 0:00:00z Length of rerun : 18,0,0,0,0,0 (Y,M,D,h,m,s) End time : l8510 = 2018/ 5/ 1 0:00:00z New run_target_end : 29,5,0,0,0,0 (Y,M,D,h,m,s) New contents of /home/rcerezo/precis/precis_expt/aabab/lbc_names: /home/rcerezo/precis/lbc/aabab/eraina.baxxxxx $LBC_DIR/eraina.bak0510 Copy of original phist file : /home/rcerezo/precis/precis_expt/aabab/aabab.phist.k05t0.45796 Copy of original phist file : /home/rcerezo/precis/tmp/aabab.phist.k05t0.45796 New phist file for time k0510: /home/rcerezo/precis/precis_expt/aabab/aabab.phist Copy of new phist file : /home/rcerezo/precis/precis_expt/aabab/aabab.phist.k0510 Copy of new phist file : /home/rcerezo/precis/tmp/aabab.phist.k0510 Restarting with command : /home/rcerezo/precis/um/vn4.5/utils/restart aabab Copy of log file : /home/rcerezo/precis/precis_expt/aabab/aabab.rerun.k0510.45796 -------------------------------------------------------------------------------- TYPE=CRUN MODEL=la365-226-155-0.44-0.44-35.65-303.6-75.74-113.98-4-19-s TYPE=CRUN: Appending to file /home/rcerezo/precis/precis_expt/aabab/aabab.time /bin/ksh: /home/rcerezo/umui_runs/aabab-208093502/SCRIPT[302]: cat: /home/rcerezo/precis/tmp/aabab.time: No such file or directory rm: cannot remove ‘/home/rcerezo/precis/tmp/aabab.time’: No such file or directory A runtime process is already running: Not running runtime Background runtime process has PID=9087 Argument "" isn't numeric in numeric ge (>=) at /home/rcerezo/precis/um/vn4.5/scripts/ancil_update line 342. Argument "" isn't numeric in numeric ge (>=) at /home/rcerezo/precis/um/vn4.5/scripts/ancil_update line 342. Argument "" isn't numeric in numeric ge (>=) at /home/rcerezo/precis/um/vn4.5/scripts/ancil_update line 342. Argument "" isn't numeric in numeric ge (>=) at /home/rcerezo/precis/um/vn4.5/scripts/ancil_update line 342. -------------------------------------------------------------------------------- ANCILLARY FILE UPDATING RUNNING SCRIPT WITH ARGUMENTS: /home/rcerezo/precis/um/vn4.5/scripts/ancil_update lun jul 27 11:49:06 CDT 2015 -------------------------------------------------------------------------------- Current model time = 2000/ 6/ 1 0:00:00z Created soft links in /home/rcerezo/precis/precis_expt/aabab/aabab_ancil_master to files in /home/rcerezo/precis/ancil/master Ancillary configuration file: /home/rcerezo/precis/um/vn4.5/ctldata/ancil_config/erain Contents: source_ozone=ozone.echvc source_vegsoil=qrparm.whs source_igbp=qrdata.igbp source_orog=qrorog.navy10.y84n source_oxi=oxi.erain source_sst=sst.erain update_types=sst,sice,ozone,so2dms source_sice=sice.erain source_whs=qrparm.whs source_soil=qrdata.soil source_volc=volc.erain source_soilcarbon=qrparm.soilcarbon source_mask=qrfrac.navy10_BAS source_so2dms=so2dms.erain source_rough=qrparm.ashhr The following ancillaries are to be updated: sst, sice, ozone, so2dms sst : Last update time = Current time = 2000/ 6/ 1 0:00:00z Update time = => sst ancillary update required Moved file /home/rcerezo/precis/ancil/aabab/qrclim.sst.erain to /home/rcerezo/precis/archive/aabab/ancil/qrclim.sst.erain.k0110 New master file /home/rcerezo/precis/ancil/master/sst.erain.k0110 spans 2000/ 1/ 1 0:00:00z to 2000/12/31 18:00:00z Soft link /home/rcerezo/precis/precis_expt/aabab/aabab_ancil_master/sst.erain -> /home/rcerezo/precis/ancil/master/sst.erain.k0110 Next update time = 2000/ 1/ 1 0:00:00z sice : Last update time = Current time = 2000/ 6/ 1 0:00:00z Update time = => sice ancillary update required Moved file /home/rcerezo/precis/ancil/aabab/qrclim.sice.erain to /home/rcerezo/precis/archive/aabab/ancil/qrclim.sice.erain.k0110 New master file /home/rcerezo/precis/ancil/master/sice.erain.k0110 spans 2000/ 1/ 1 0:00:00z to 2000/12/31 18:00:00z Soft link /home/rcerezo/precis/precis_expt/aabab/aabab_ancil_master/sice.erain -> /home/rcerezo/precis/ancil/master/sice.erain.k0110 Next update time = 2000/ 1/ 1 0:00:00z ozone : Last update time = Current time = 2000/ 6/ 1 0:00:00z Update time = => ozone ancillary update required Moved file /home/rcerezo/precis/ancil/aabab/qrclim.ozone.erain to /home/rcerezo/precis/archive/aabab/ancil/qrclim.ozone.erain.h9bg0 New master file /home/rcerezo/precis/ancil/master/ozone.echvc.h9c10 spans 1979/11/16 0:00:00z to 2009/12/16 0:00:00z Soft link /home/rcerezo/precis/precis_expt/aabab/aabab_ancil_master/ozone.echvc -> /home/rcerezo/precis/ancil/master/ozone.echvc.h9c10 Next update time = 1979/11/16 0:00:00z so2dms : Last update time = Current time = 2000/ 6/ 1 0:00:00z Update time = => so2dms ancillary update required Moved file /home/rcerezo/precis/ancil/aabab/qrclim.so2dms.erain to /home/rcerezo/precis/archive/aabab/ancil/qrclim.so2dms.erain.e9910 New master file /home/rcerezo/precis/ancil/master/so2dms.erain.e9c10 spans 1949/ 9/ 1 0:00:00z to 2000/ 6/ 1 0:00:00z Soft link /home/rcerezo/precis/precis_expt/aabab/aabab_ancil_master/so2dms.erain -> /home/rcerezo/precis/ancil/master/so2dms.erain.e9c10 Next update time = 1949/ 9/ 1 0:00:00z Updated ancillaries will be created from master files. Create new ancillary files: -------------------------------------------------------------------------------- ANCILLARY FILE CREATION RUNNING SCRIPT WITH ARGUMENTS: /home/rcerezo/precis/um/vn4.5/utils/ancil_create -a /home/rcerezo/precis/ancil/aabab -b /home/rcerezo/precis/precis_expt/aabab/aabab_ancil_master -c /home/rcerezo/precis/um/vn4.5/ctldata/ancil_config/erain -d /home/rcerezo/umui_runs/aabab-208093502 -o /home/rcerezo/umui_runs/aabab-208093502/override_orogmask -r /home/rcerezo/umui_runs/aabab-208093502/override -t /home/rcerezo/umui_runs/aabab-208093502/override_mean_orog -u /home/rcerezo/umui_runs/aabab-208093502/override_vegsoil erain lun jul 27 11:49:07 CDT 2015 -------------------------------------------------------------------------------- Grid namelist vn4.4 /home/rcerezo/precis/ancil/aabab/gridnamelist_vn4.4 has been created from files in /home/rcerezo/umui_runs/aabab-208093502 Grid namelist vn5.0 /home/rcerezo/precis/ancil/aabab/gridnamelist_vn5.0 has been created from /home/rcerezo/precis/ancil/aabab/gridnamelist_vn4.4 Grid namelist vn7.4 /home/rcerezo/precis/ancil/aabab/gridnamelist_vn7.4 has been created from files in /home/rcerezo/umui_runs/aabab-208093502 Ancillary configuration file: /home/rcerezo/precis/um/vn4.5/ctldata/ancil_config/erain Contents: source_orog=qrorog.navy10.y84n source_sst=sst.erain source_soil=qrdata.soil source_so2dms=so2dms.erain source_whs=qrparm.whs source_ozone=ozone.echvc source_igbp=qrdata.igbp source_sice=sice.erain source_volc=volc.erain source_soilcarbon=qrparm.soilcarbon update_types=sst,sice,ozone,so2dms source_vegsoil=qrparm.whs source_mask=qrfrac.navy10_BAS source_rough=qrparm.ashhr source_oxi=oxi.erain mask : /home/rcerezo/precis/ancil/aabab/qrparm.mask exists. Not recreated. /home/rcerezo/precis/ancil/aabab/qrparm.mask spans 0/ 0/ 0 0:00:00z to 0/ 0/ 0 0:00:00z orog : /home/rcerezo/precis/ancil/aabab/qrparm.orog exists. Not recreated. /home/rcerezo/precis/ancil/aabab/qrparm.orog spans 0/ 0/ 0 0:00:00z to 0/ 0/ 0 0:00:00z vegsoil : Not required. Not created. MOSES2 : /home/rcerezo/precis/ancil/aabab/qrparm.veg.frac and/or /home/rcerezo/precis/ancil/aabab/qrparm.veg.func and/or /home/rcerezo/precis/ancil/aabab/qrparm.soil.MOSES_IGBP and/or /home/rcerezo/precis/ancil/aabab/qrparm.veg.dist exist. Not recreated. /home/rcerezo/precis/ancil/aabab/qrparm.veg.frac spans 0/ 0/ 0 0:00:00z to 0/ 0/ 0 0:00:00z /home/rcerezo/precis/ancil/aabab/qrparm.veg.func spans 0/ 0/ 0 0:00:00z to 0/ 0/ 0 0:00:00z /home/rcerezo/precis/ancil/aabab/qrparm.soil.MOSES_IGBP spans 0/ 0/ 0 0:00:00z to 0/ 0/ 0 0:00:00z /home/rcerezo/precis/ancil/aabab/qrparm.veg.dist spans 0/ 0/ 0 0:00:00z to 0/ 0/ 0 0:00:00z sstice : /home/rcerezo/precis/ancil/aabab/qrclim.sice.erain /home/rcerezo/precis/ancil/aabab/qrclim.sst.erain created successfully. See /home/rcerezo/precis/ancil/aabab/.aabab.sstice.out for details. /home/rcerezo/precis/ancil/aabab/qrclim.sice.erain spans 2000/ 1/ 1 0:00:00z to 2000/12/31 18:00:00z /home/rcerezo/precis/ancil/aabab/qrclim.sst.erain spans 2000/ 1/ 1 0:00:00z to 2000/12/31 18:00:00z ozone : /home/rcerezo/precis/ancil/aabab/qrclim.ozone.erain created successfully. See /home/rcerezo/precis/ancil/aabab/.aabab.ozone.out for details. /home/rcerezo/precis/ancil/aabab/qrclim.ozone.erain spans 1979/11/16 0:00:00z to 2009/12/16 0:00:00z so2dms : /home/rcerezo/precis/ancil/aabab/qrclim.so2dms.erain created successfully. See /home/rcerezo/precis/ancil/aabab/.aabab.so2dms.out for details. /home/rcerezo/precis/ancil/aabab/qrclim.so2dms.erain spans 1949/ 9/ 1 0:00:00z to 2000/ 6/ 1 0:00:00z oxi : /home/rcerezo/precis/ancil/aabab/qrclim.oxi.erain exists. Not recreated. /home/rcerezo/precis/ancil/aabab/qrclim.oxi.erain spans 1990/ 1/16 0:00:00z to 1990/12/16 0:00:00z volc : /home/rcerezo/precis/ancil/aabab/qrparm.volc.erain exists. Not recreated. /home/rcerezo/precis/ancil/aabab/qrparm.volc.erain spans 1991/ 1/ 1 0:00:00z to 1991/ 1/ 1 0:00:00z Final contents of ANCIL_DIR= /home/rcerezo/precis/ancil/aabab total 1664044 drwxrwxr-x 2 rcerezo 4096 jul 27 11:49 . drwxrwxr-x 5 rcerezo 4096 jul 9 15:49 .. -rw-rw-r-- 1 rcerezo 62058 jul 9 15:49 .aabab.mask.out -rw-rw-r-- 1 rcerezo 913340 jul 9 15:51 .aabab.MOSES2.out -rw-rw-r-- 1 rcerezo 334776 jul 9 15:49 .aabab.orog.out -rw-rw-r-- 1 rcerezo 198445 jul 9 15:52 .aabab.oxi.out -rw-rw-r-- 1 rcerezo 762843 jul 27 11:49 .aabab.ozone.out -rw-rw-r-- 1 rcerezo 197073 jul 27 11:49 .aabab.so2dms.out -rw-rw-r-- 1 rcerezo 845446 jul 27 11:49 .aabab.sstice.out -rw-rw-r-- 1 rcerezo 114564 jul 9 15:52 .aabab.volc.out -rw-rw-r-- 1 rcerezo 357 jul 27 11:49 gridnamelist_vn4.4 -rw-rw-r-- 1 rcerezo 324 jul 27 11:49 gridnamelist_vn5.0 -rw-rw-r-- 1 rcerezo 255 jul 27 11:49 gridnamelist_vn7.4 -rw-rw-r-- 1 rcerezo 656 jul 9 15:49 .make_ancil_mask.namelist -rwxr-xr-x 1 rcerezo 1699 jul 9 15:49 .make_ancil_mask.script -rw-rw-r-- 1 rcerezo 2018 jul 9 15:49 .make_ancil_MOSES2.namelist -rw-rw-r-- 1 rcerezo 5213 jul 9 15:49 .make_ancil_MOSES2.script -rw-rw-r-- 1 rcerezo 812 jul 9 15:49 .make_ancil_orog.namelist -rwxr-xr-x 1 rcerezo 1880 jul 9 15:49 .make_ancil_orog.script -rw-rw-r-- 1 rcerezo 504 jul 9 15:52 .make_ancil_oxi.namelist -rw-rw-r-- 1 rcerezo 1650 jul 9 15:52 .make_ancil_oxi.script -rw-rw-r-- 1 rcerezo 504 jul 27 11:49 .make_ancil_ozone.namelist -rw-rw-r-- 1 rcerezo 1656 jul 27 11:49 .make_ancil_ozone.script -rw-rw-r-- 1 rcerezo 504 jul 27 11:49 .make_ancil_so2dms.namelist -rw-rw-r-- 1 rcerezo 1659 jul 27 11:49 .make_ancil_so2dms.script -rw-rw-r-- 1 rcerezo 657 jul 27 11:49 .make_ancil_sstice.namelist -rwxr-xr-x 1 rcerezo 1828 jul 27 11:49 .make_ancil_sstice.script -rw-rw-r-- 1 rcerezo 504 jul 9 15:52 .make_ancil_volc.namelist -rw-rw-r-- 1 rcerezo 1653 jul 9 15:52 .make_ancil_volc.script -rw-rw-r-- 1 rcerezo 459 jul 9 15:49 .overridefile_mask -rw-rw-r-- 1 rcerezo 134717440 jul 9 15:52 qrclim.oxi.erain -rw-rw-r-- 1 rcerezo 1015971840 jul 27 11:49 qrclim.ozone.erain -rw-rw-r-- 1 rcerezo 216245080 jul 27 11:49 qrclim.sice.erain -rw-rw-r-- 1 rcerezo 120537088 jul 27 11:49 qrclim.so2dms.erain -rw-rw-r-- 1 rcerezo 216245080 jul 27 11:49 qrclim.sst.erain -rw-rw-r-- 1 rcerezo 148312 jul 9 15:49 qrparm.mask -rw-rw-r-- 1 rcerezo 1033048 jul 9 15:49 qrparm.orog -rw-rw-r-- 1 rcerezo 148312 jul 9 15:49 qrparm.orog.mask -rw-rw-r-- 1 rcerezo 1630208 jul 9 15:51 qrparm.soil.MOSES_IGBP -rw-rw-r-- 1 rcerezo 155648 jul 9 15:51 qrparm.veg.dist -rw-rw-r-- 1 rcerezo 1335296 jul 9 15:51 qrparm.veg.frac -rw-rw-r-- 1 rcerezo 1482752 jul 9 15:51 qrparm.veg.func -rw-rw-r-- 1 rcerezo 2809856 jul 9 15:52 qrparm.volc.erain -------------------------------------------------------------------------------- Ancillary creation completed at lun jul 27 11:49:39 CDT 2015 -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- Ancillary updating completed at lun jul 27 11:49:39 CDT 2015 -------------------------------------------------------------------------------- Time taken by /home/rcerezo/precis/um/vn4.5/scripts/ancil_update: **ANCIL_TIME: 0m33.14s real 0m27.67s user 0m5.44s system -------------------------------------------------------------------------------- LATERAL BOUNDARY CONDITION UPDATING RUNNING SCRIPT WITH ARGUMENTS: /home/rcerezo/precis/um/vn4.5/scripts/lbc_update lun jul 27 11:49:39 CDT 2015 -------------------------------------------------------------------------------- TYPE = CRUN Restart time = 2000/ 6/ 1 0:00:00z Next update time = 2000/ 6/ 1 0:00:00z (k0610) LBC file /home/rcerezo/precis/lbc/aabab/eraina.bak0610 exists and has non-zero size. Deleted LBC file /home/rcerezo/precis/lbc/aabab/eraina.bak0610 Creating LBC file /home/rcerezo/precis/lbc/aabab/eraina.bak0610. Details: Using standard prognostics PP4LBC file /home/rcerezo/precis/pp4lbc/erain/eraina.CORDEX_CentralAmerica.pik0610 See log file /home/rcerezo/precis/precis_expt/aabab/lbc/eraina.bak0610.out LBC file /home/rcerezo/precis/lbc/aabab/eraina.bak0610 exists and has non-zero size. Created soft link: /home/rcerezo/precis/precis_expt/aabab/alabcin_aabab -> /home/rcerezo/precis/lbc/aabab/eraina.bak0610 Deleted LBC file: /home/rcerezo/precis/lbc/aabab/eraina.bak0510 Current and future LBC files: /home/rcerezo/precis/lbc/aabab/eraina.bak0610 $LBC_DIR/eraina.bak0710 -------------------------------------------------------------------------------- Lateral boundary condition updating completed at lun jul 27 11:49:41 CDT 2015 -------------------------------------------------------------------------------- Time taken by /home/rcerezo/precis/um/vn4.5/scripts/lbc_update: **LBC_TIME: 0m1.69s real 0m0.78s user 0m0.70s system PATH USED = /home/rcerezo/precis//cdat-3.3/bin:/home/rcerezo/precis/um/ui:/home/rcerezo/precis/um/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/rcerezo/precis/um/vn4.5/utils:/home/rcerezo/precis/um/bin/mpich2/bin:/home/rcerezo/precis/um/rove:/home/rcerezo/precis/tmp/modscr_aabab:/vn4.5/exec::/u/m20/home/mec/t20sm/rove:/home/rcerezo/precis/um/vn4.5/scripts:/home/rcerezo/precis/um/vn4.5/exec ***************************************************************** Starting script : qsprelim Starting time : lun jul 27 11:49:41 CDT 2015 ***************************************************************** /home/rcerezo/precis/um/vn4.5/scripts/qsprelim: Executing combine /home/rcerezo/precis/um/vn4.5/scripts/qscombine: Job terminated normally qsprelim: /home/rcerezo/precis/precis_expt/aabab/aabab.thist_keep moved to /home/rcerezo/precis/precis_expt/aabab/aabab.thist ***************************************************************** Ending script : qsprelim Completion code : 0 Completion time : lun jul 27 11:49:41 CDT 2015 ***************************************************************** ***************************************************************** Starting script : qsmain Starting time : lun jul 27 11:49:41 CDT 2015 ***************************************************************** aabab: Starting run hello -machinefile /home/rcerezo/precis/tmp/machines.208093502 -np 16 /home/rcerezo/precis/precis_expt/aabab/aabab.parexe ===================================================== GCOM Version 3.0 MPI Using precision : 32bit INTEGERs and 32bit REALs Built at Wed Apr 30 10:34:12 BST 2008 ===================================================== OPEN: File /home/rcerezo/precis/precis_expt/aabab/aababa.dak0610 to be Opened on Unit 21 Exists OPEN: File /home/rcerezo/precis/precis_expt/aabab/aababa.dak0610 to be Opened on Unit 21 Exists OPEN: File /home/rcerezo/precis/ancil/aabab/qrclim.ozone.erain to be Opened on Unit 30 Exists OPEN: File /home/rcerezo/precis/ancil/aabab/qrclim.sst.erain to be Opened on Unit 35 Exists OPEN: File /home/rcerezo/precis/ancil/aabab/qrclim.sice.erain to be Opened on Unit 36 Exists OPEN: File /home/rcerezo/precis/ancil/aabab/qrclim.so2dms.erain to be Opened on Unit 110 Exists OPEN: File /home/rcerezo/precis/ancil/aabab/qrclim.oxi.erain to be Opened on Unit 116 Exists OPEN: File /home/rcerezo/precis/precis_expt/aabab/alabcin_aabab to be Opened on Unit 95 Exists OPEN: File /home/rcerezo/precis/precis_expt/aabab/aababa.pak0may to be Opened on Unit 60 Exists OPEN: File /home/rcerezo/precis/precis_expt/aabab/aababa.pjk0610 to be Opened on Unit 69 Exists ***************************************** ***************************************** Model completed with the following : Error Code : 539 Message : REPLANCA: TIME INTERPOLATION ERROR ***************************************** ***************************************** forrtl: error (76): IOT trap signal forrtl: error (76): IOT trap signal forrtl: error (76): IOT trap signal forrtl: error (76): IOT trap signal forrtl: error (76): IOT trap signal forrtl: error (76): IOT trap signal forrtl: error (76): IOT trap signal forrtl: error (76): IOT trap signal forrtl: error (76): IOT trap signal forrtl: error (76): IOT trap signal forrtl: error (76): IOT trap signal forrtl: error (76): IOT trap signal forrtl: error (76): IOT trap signal forrtl: error (76): IOT trap signal forrtl: error (76): IOT trap signal forrtl: error (76): IOT trap signal Aborted (core dumped) rank 5 in job 143 UAS1S0P000A160U_47705 caused collective abort of all ranks exit status of rank 5: killed by signal 9 0m1.24s real 0m0.07s user 0m0.01s system qsmain: Time taken by model executable /home/rcerezo/precis/um/vn4.5/exec/hadrm3p.MOSESII.exec.mpp: **MODEL_TIME: qsmain: LOADMODULE CC=137 diff: /home/rcerezo/precis/tmp/aabab.xhist: No such file or directory qsmain: Copying /home/rcerezo/precis/precis_expt/aabab/aabab.thist to backup thist file /home/rcerezo/precis/precis_expt/aabab/aabab.thist_keep aabab: Run failed aabab: Information on which CPU was in use follows:- aabab: debug information (if activated) follows :- /home/rcerezo/precis/um/vn4.5/scripts/qsmain[1010]: debugview: not found core file copied to /home/rcerezo/precis/tmp/aabab.core ***************************************************************** Ending script : qsmain Completion code : 137 Completion time : lun jul 27 11:49:44 CDT 2015 ***************************************************************** /home/rcerezo/precis/um/vn4.5/scripts/qsmaster: failed in main in model aabab ***************************************************************** Starting script : qsfinal Starting time : lun jul 27 11:49:44 CDT 2015 ***************************************************************** qsfinal: thist file copied to /home/rcerezo/precis/precis_expt/aabab/aabab.thist.9427 /home/rcerezo/precis/um/vn4.5/scripts/qsfinal: Error in exit processing after model run Failed in model executable /home/rcerezo/precis/um/vn4.5/scripts/qspickup: Normal completion /home/rcerezo/precis/um/vn4.5/scripts/qshistprint: Job terminated normally /home/rcerezo/precis/um/vn4.5/scripts/qsresubmit: Error job not resubmitted because of error in qsmain ***************************************************************** Ending script : qsfinal Completion code : 135 Completion time : lun jul 27 11:49:45 CDT 2015 ***************************************************************** /home/rcerezo/precis/um/vn4.5/scripts/qsmaster: failed in final in model aabab <<<< Information about How Many Lines of Output follow >>>> 7997 lines in main OUTPUT file. 22 lines in SERVER file. <<<< Lines of Output Information ends >>>> OUTPUT 5394 lines of output from pe0. PE0 OUTPUT %PE0 OUTPUT% 16 Processors initialised. I am PE 0 Start of UM Job : 11:49:42 on 27/07/2015 Setting UM_NAM_MAX_SECONDS to 300 UMSETUP; CRUN, read CONTCNTL from unit 9 Number of levels for convective clouds is 19 global_land_field set to 10321 PE 0 : LAND_FIELD is being reset from 10321 to 1505 local_land_field set to 1505 n_intf_a 0 n_intf_a 1 tot_len_intfa_p 1 tot_len_intfa_u 1 max_intf_p_levels 1 u_field_intfa 1 !!!! STASH_MSTR /home/rcerezo/precis/um/vn4.5/ctldata/STASHmaster/STASHmaster_A MESSAGE FROM ROUTINE GETPPX: The following PPXREF record has been overwritten by a record read from a user-STASH master file: Internal Model 1 Section 0 Item 229 MESSAGE FROM ROUTINE GETPPX: The following PPXREF record has been overwritten by a record read from a user-STASH master file: Internal Model 1 Section 0 Item 230 MESSAGE FROM ROUTINE GETPPX: The following PPXREF record has been overwritten by a record read from a user-STASH master file: Internal Model 1 Section 0 Item 231 MESSAGE FROM ROUTINE GETPPX: The following PPXREF record has been overwritten by a record read from a user-STASH master file: Internal Model 1 Section 3 Item 314 MESSAGE FROM ROUTINE GETPPX: The following PPXREF record has been overwritten by a record read from a user-STASH master file: Internal Model 1 Section 3 Item 321 MESSAGE FROM ROUTINE GETPPX: The following PPXREF record has been overwritten by a record read from a user-STASH master file: Internal Model 1 Section 3 Item 322 MESSAGE FROM ROUTINE GETPPX: The following PPXREF record has been overwritten by a record read from a user-STASH master file: Internal Model 1 Section 3 Item 287 MESSAGE FROM ROUTINE GETPPX: The following PPXREF record has been overwritten by a record read from a user-STASH master file: Internal Model 1 Section 3 Item 288 MESSAGE FROM ROUTINE GETPPX: The following PPXREF record has been overwritten by a record read from a user-STASH master file: Internal Model 1 Section 0 Item 101 MESSAGE FROM ROUTINE GETPPX: The following PPXREF record has been overwritten by a record read from a user-STASH master file: Internal Model 1 Section 0 Item 102 MESSAGE FROM ROUTINE GETPPX: The following PPXREF record has been overwritten by a record read from a user-STASH master file: Internal Model 1 Section 0 Item 103 MESSAGE FROM ROUTINE GETPPX: The following PPXREF record has been overwritten by a record read from a user-STASH master file: Internal Model 1 Section 0 Item 104 MESSAGE FROM ROUTINE GETPPX: The following PPXREF record has been overwritten by a record read from a user-STASH master file: Internal Model 1 Section 0 Item 105 MESSAGE FROM ROUTINE GETPPX: The following PPXREF record has been overwritten by a record read from a user-STASH master file: Internal Model 1 Section 0 Item 106 MESSAGE FROM ROUTINE GETPPX: The following PPXREF record has been overwritten by a record read from a user-STASH master file: Internal Model 1 Section 0 Item 107 PRELIM: START TIME BEFORE PERIOD, SETTING TO 1 MODEL,SECTION,ITEM 1 1 201 PRELIM: START TIME BEFORE PERIOD, SETTING TO 1 MODEL,SECTION,ITEM 1 1 235 PRELIM: START TIME BEFORE PERIOD, SETTING TO 1 MODEL,SECTION,ITEM 1 2 201 PRELIM: START TIME BEFORE PERIOD, SETTING TO 1 MODEL,SECTION,ITEM 1 2 204 MESSAGE FROM ROUITNE PRELIM: DIAGNOSTIC REQUEST HAS LEVEL RANGE OUT OF BOUNDS; HAS BEEN CORRECTED MODEL,SECTION,ITEM 1 3 219 MESSAGE FROM ROUTINE PRELIM: DIAGNOSTIC REQUEST HAS TOP/BOTTOM LEVELS INCONSISTENT; REQUEST IGNORED MODEL,SECTION,ITEM 1 3 219 MESSAGE FROM ROUITNE PRELIM: DIAGNOSTIC REQUEST HAS LEVEL RANGE OUT OF BOUNDS; HAS BEEN CORRECTED MODEL,SECTION,ITEM 1 3 220 MESSAGE FROM ROUTINE PRELIM: DIAGNOSTIC REQUEST HAS TOP/BOTTOM LEVELS INCONSISTENT; REQUEST IGNORED MODEL,SECTION,ITEM 1 3 220 MESSAGE FROM ROUTINE PRELIM: DIAGNOSTIC REQUEST HAS INVALID PSEUDO LEVEL TYPE; REQUEST IGNORED MODEL,SECTION,ITEM 1 3 293 OPEN: 3: /home/rcerezo/precis/um/vn4.5/ctldata/stasets/X01001218 : FILE EXISTS OPEN: 3: /home/rcerezo/precis/um/vn4.5/ctldata/stasets/X01003274 : FILE EXISTS OPEN: 3: /home/rcerezo/precis/um/vn4.5/ctldata/stasets/X01003275 : FILE EXISTS OPEN: 3: /home/rcerezo/precis/um/vn4.5/ctldata/stasets/X01003276 : FILE EXISTS OPEN: 3: /home/rcerezo/precis/um/vn4.5/ctldata/stasets/X01003277 : FILE EXISTS OPEN: 3: /home/rcerezo/precis/um/vn4.5/ctldata/stasets/X01003278 : FILE EXISTS OPEN: 3: /home/rcerezo/precis/um/vn4.5/ctldata/stasets/X01003279 : FILE EXISTS OPEN: 3: /home/rcerezo/precis/um/vn4.5/ctldata/stasets/X01003280 : FILE EXISTS OPEN: 3: /home/rcerezo/precis/um/vn4.5/ctldata/stasets/X01003281 : FILE EXISTS OPEN: 3: /home/rcerezo/precis/um/vn4.5/ctldata/stasets/X01003286 : FILE EXISTS OPEN: 3: /home/rcerezo/precis/um/vn4.5/ctldata/stasets/X01005207 : FILE EXISTS OPEN: 3: /home/rcerezo/precis/um/vn4.5/ctldata/stasets/X01005208 : FILE EXISTS OPEN: 3: /home/rcerezo/precis/um/vn4.5/ctldata/stasets/X01005222 : FILE EXISTS OPEN: 3: /home/rcerezo/precis/um/vn4.5/ctldata/stasets/X01005223 : FILE EXISTS OPEN: 3: /home/rcerezo/precis/um/vn4.5/ctldata/stasets/X01010206 : FILE EXISTS ADDRES : A_PROG_LOOKUP = 488 ADDRES : A_PROG_LEN = 1179656 ADDRES : S_PROG_LOOKUP = 0 ADDRES : S_PROG_LEN = 0 ADDRES : O_PROG_LOOKUP = 0 ADDRES : O_PROG_LEN = 0 ADDRES : W_PROG_LOOKUP = 0 ADDRES : W_PROG_LEN = 0 285 D1 items in submodel 1 323 STASH LIST RECORDS Free disk space: Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda4 4054032664 776648104 3071428516 21% /home **************************************************************** Job ended at : lun jul 27 11:49:45 CDT 2015 **************************************************************** Killed the background runtime process with PID=9087 runtime output at end of run: ------------------------------------------------------+------------------------- REAL TIME : 11:49:45 CDT lunes 27/jul/2015 | RATE : 1467588.25 months/day MODELS : HadRM3P (MOSES2.2), ERA-Interim (erain) | TIME : 13 seconds (end) RUNID : aabab | DATE : 11:49 27/Jul/2015 STATUS : CRUN finished at 11:49 27/Jul/2015 | MWLE : 0% ------------------------------+-----------------------+------------------------- start : 1988/12/ 1 0:00z | COMPLETED : 11 Years 5 Months 28 Days CURRENT : 2000/ 5/29 0:00z | % : 39.067% of 29 Years 5 Months end : 2018/ 5/ 1 0:00z | TIMESTEPS : 2417472/6187968 (150 secs/TS) ------------------------------+------------------------------------------------- MWLE, timestep and time information from /home/rcerezo/precis/precis_expt/aabab/aabab.time: # Col 1: Output file # Col 2: Model time at start of file # Col 3: Model time at end of file # Col 4: No. of MWLE timesteps in output file # Col 5: No. of MWLE timesteps in whole run # Col 6: No. of timesteps in output file # Col 7: First timestep in output file # Col 8: Last timestep in output file # Col 9: Number of real seconds for CRUN # Col 10: Number of real seconds for run so far /home/rcerezo/precis/precis_out/aabab000.aabab.d15190.t154917.leave , 1988/12/ 1 , 1989/ 1/ 1 , 0 , 0 , 8928 , 1 , 8928 , 3 , 3 /home/rcerezo/precis/precis_out/aabab000.aabab.d15191.t103545.leave , 1989/ 1/ 1 , 1989/ 2/ 1 , 0 , 0 , 8928 , 8929 , 17856 , 0 , 3 /home/rcerezo/precis/precis_out/aabab000.aabab.d15195.t152248.leave , 1993/ 2/ 1 , 1993/ 3/ 1 , 0 , 0 , 8064 , 438625 , 446688 , 0 , 3 /home/rcerezo/precis/precis_out/aabab000.aabab.d15208.t093250.leave , 1988/12/ 0 , 1988/12/ 1 , 0 , 0 , 0 , 0 , 0 , 0 , 3 /home/rcerezo/precis/precis_out/aabab000.aabab.d15208.t093502.leave Executing exit commands: kill -9 9087 ; rm -f /home/rcerezo/.jobrunning.aabab.9055 /home/rcerezo/precis/tmp/aabab.runtime_info /home/rcerezo/precis/tmp/aabab.runtime_rate /bin/ksh: kill: 9087: No such process |
Author: | sotucker [ Mon Aug 03, 2015 12:10 pm ] |
Post subject: | Re: Error Code : 539 Message : REPLANCA: |
Hi Ruth, Did you try using the rerun command after the model had already crashed? I don't know if this is relevant, but in more recent PRECIS versions we have disabled the CRUN option from the rerun command because it was error prone. Are you able to try submitting as an NRUN? i.e. home/rcerezo/precis/um/vn4.5/utils/rerun aabab k0510 18,0,0,0,0,0 NRUN , and if the model successfully runs for a month restart aabab? All the best, Simon |
Author: | RuthCM [ Thu Aug 27, 2015 5:28 pm ] | ||
Post subject: | Re: Error Code : 539 Message : REPLANCA: | ||
Hi! I restarted the full simulation, since I got a new ERA-Interim dataset (starting in 1979), and everything run smoothly until the simulation reached May 2000. It is sending the same error message: REPLANCA: TIME INTERPOLATION ERROR I have tried to restart the model from 1st May 2000 (with rerun k0510...NRUN) and it did run for the full month then stop (as normal) if I type: restart idrun, the model tried to restart but then crashed with the same error then I tried to restart from 1st June 2000(since the model have that start dump), again using rerun k0610 ...NRUN and it crashed and with CRUN the same... I attached here the last leave file...help!
|
Author: | sotucker [ Wed Sep 02, 2015 2:17 pm ] |
Post subject: | Re: Error Code : 539 Message : REPLANCA: |
Dear Ruth, To restart your simulation again, you simply need to unzip the file $ANCIL_MASTER/so2dms.erain.k0310.gz I am afraid however that an error has been found in the SO2DMS ancillaries that you are using. Specifically the DMS fields contain no emissions over the oceans. It is unlikely that the error will have a large affect on your simulation, however please contact us should you wish to rerun with a corrected ancillary. Regards, Simon |
Author: | RuthCM [ Thu Sep 03, 2015 2:57 pm ] |
Post subject: | Re: Error Code : 539 Message : REPLANCA: |
Hi Simon, Thanks for the reply, but I am not so sure that I understood. Do you mean that all the DMS files that have been used on this simulation have an error ? including this one that I have just unzip? I am no so sure I will re-do the simulation now, but it will be good to have the right files (just in case) and probably now that I have to restart the simulation to use the right one? How can I get the correct one? cheers, Ruth |
Author: | sotucker [ Wed Sep 09, 2015 10:20 am ] |
Post subject: | Re: Error Code : 539 Message : REPLANCA: |
Hi Ruth, Sorry, I thought that I replied to your message on Monday. Replacement ancillaries can be downloaded from http://www.precisrcm.com/Ruth/. To use these you will need to edit the file $UMDIR/vn4.5/ctldata/ancil_config/erain by replacing the line Quote: source_so2dms = so2dms.erain with Quote: source_so2dms = so2dms.historical.greg
|
Page 1 of 1 | All times are UTC |
Powered by phpBB® Forum Software © phpBB Group https://www.phpbb.com/ |