Commit graph

23 commits

Author SHA1 Message Date
Nathan Binkert
f82d4e2364 python: Move various utility classes into a new m5.util package so
they're all in the same place.  This also involves having just one
jobfile.py and moving it into the utils directory to avoid
duplication.  Lots of improvements to the utility as well.

--HG--
rename : src/python/m5/attrdict.py => src/python/m5/util/attrdict.py
rename : util/pbs/jobfile.py => src/python/m5/util/jobfile.py
rename : src/python/m5/util.py => src/python/m5/util/misc.py
rename : src/python/m5/multidict.py => src/python/m5/util/multidict.py
rename : util/stats/orderdict.py => src/python/m5/util/orderdict.py
2008-06-14 20:19:49 -07:00
Nathan Binkert
f462266b30 don't add an empty suboption description
--HG--
extra : convert_revision : 594744c3d438aed08a23db376959930071b2c368
2006-02-23 00:21:35 -05:00
Nathan Binkert
35094fb0fe make it possible to add filters for job names so that
parts of the full crossproduct of jobs can be ignored.

--HG--
extra : convert_revision : c44b3daea0cf4b487b1d99eae92da16573b15930
2006-02-23 00:20:32 -05:00
Ali Saidi
3b66cb49ec Merge zizzer:/bk/m5
into  zeep.eecs.umich.edu:/z/saidi/work/m5

--HG--
extra : convert_revision : 3cc23080d19cc464a8ba7c1c93b6e5d45af7d463
2005-11-02 14:56:18 -05:00
Ali Saidi
4b6540c264 add a few more options to the help menu
--HG--
extra : convert_revision : b2481bedac786e4a6bb0d577954242d7f4c144a0
2005-11-02 14:54:21 -05:00
Nathan Binkert
a0829a7780 Simple updates to pbs and send.py
util/pbs/pbs.py:
    Change the default so that we do not get mail under any circumstances
    from pbs.
util/pbs/send.py:
    Add a -n flag to send.py that causes the Base directory to *not*
    sync with the Link directory

--HG--
extra : convert_revision : 6e872153b6b2c34b61ec2ddbf3e5536876f4b43b
2005-11-02 12:19:08 -05:00
Nathan Binkert
8ab674582e fix pbs dependency stuff
util/pbs/pbs.py:
    after -> afterok

--HG--
extra : convert_revision : ee0af716bcc0f83c4103632bd277a5680b0d2c23
2005-10-19 02:10:24 -04:00
Nathan Binkert
357ee7a845 Better pool job handling
util/pbs/job.py:
    the default jobfile is now Test.py in the root of the jobs directory
util/pbs/pbs.py:
    Clean up the qsub options handling and add job dependencies
util/pbs/send.py:
    the default jobfile is now Test.py in the root of the jobs directory
    add a flag to depend on your checkpoint
    add a flag to specify your node type
    create the base directory if it doesn't exist

--HG--
extra : convert_revision : dfffa4a5b0e68b2550a28fbb06b9d6a208ea1f2e
2005-10-18 15:05:01 -04:00
Nathan Binkert
6bc057139d don't write the pbs jobid here, do it in send.py so we know
what it is sooner
Don't handle sigstop since you're not allowed to.

util/pbs/send.py:
    write the pbs jobid here in send.py so we know what it is sooner

--HG--
extra : convert_revision : 93292d046cb4b628031e0e57e39eb4470b598ed8
2005-10-12 13:01:55 -04:00
Nathan Binkert
02098f8e7b Totally re-work the way that jobfiles are done so there is more
information that can be used for other aspects of sending jobs.
New graphing output stuff with matplotlib.

util/pbs/job.py:
    Shuffle code around and create the JobDir class which encapsulates
    all of the functionality needed for making, organizing, and cleaning
    a job directory.
    Better status output
util/pbs/jobfile.py:
    Majory re-working of the jobfile code.
    A job file now consists of several objects that describe how
    jobs should be run, it includes information about checkpoints,
    and graphing.
util/pbs/send.py:
    use the new jobfile code.
    deal with the 15 character limit of pbs by truncating the name and
    using the raj hack.
util/stats/db.py:
    fix the __str__ function for nodes
    provide __getitem__ for the Database class
util/stats/stats.py:
    use the jobfile stuff to figure out what the proper naming
    and organziation of the graphs should be.
    move all output code to output.py, get rid of ploticus and use
    matplotlib

--HG--
rename : util/categories.py => util/stats/categories.py
extra : convert_revision : 0d793cbf6ad9492290e8ec875ce001c84095e1f7
2005-09-17 16:51:26 -04:00
Nathan Binkert
09bb203484 Updates to job scripts to accept more than 15 characters of jobname
Make the Link directory even more useful by working with
sub-directories.

util/pbs/job.py:
    Expose JOBNAME as a separate parameter from PBS_JOBNAME.  If the
    former exists, it is used as the jobname for starting the job, if
    it doesn't exist, PBS_JOBNAME is used.  This is to get around the 15
    character maximum pbs job name length.  While we're at it, shuffle
    things around to hopefully make things a bit more clear.
util/pbs/send.py:
    Make the Link directory functionality more sophisticated, copy
    sub-directories and links to directories.  (we still don't copy
    dotfiles though)
    Add the setname() function to contact pbs and use raj's hack to
    tell the webpage about longer jobnames. (it's gross, don't look)
    truncate the pbs job name to 15 characters so that it works.

--HG--
extra : convert_revision : 4a76b1a1c33721c7ca93e2fbb761f95bc3a2ac69
2005-08-16 11:27:49 -04:00
Steve Reinhardt
609cacc3da Get rid of vestiges of .mpy file handling.
--HG--
extra : convert_revision : 309b051be3473e2d42d3200c1af84227d01b5900
2005-06-04 18:41:43 -04:00
Nathan Binkert
c5b63bc4c1 Improvements to send.py to allow the user to specify the jobfile
util/pbs/send.py:
    Allow the user to specify the jobfile

--HG--
extra : convert_revision : 0e21d2b03355bb7e8938c828bbaa441dc51afd1a
2005-03-24 12:25:34 -05:00
Nathan Binkert
2b89c38172 Make it easier to find a jobfile.
util/pbs/jobfile.py:
    Search for the jobfile in sys.path

--HG--
extra : convert_revision : 50d2c2c13b6b9de4f6bc4e833961e309a98b0d2b
2005-03-07 13:05:41 -05:00
Ali Saidi
2e4bb0fc0c Merge zizzer:/bk/m5 into zeep.eecs.umich.edu:/z/saidi/work/m5
--HG--
extra : convert_revision : c807a78d9c3f3be51763dab9685aa4b7361c585c
2005-02-14 20:47:23 -05:00
Ali Saidi
f4e2f0ea33 undoing change per nates request
--HG--
extra : convert_revision : c5c2fd88dfd8d893da51c2b80907260ec14a7593
2005-02-14 20:47:05 -05:00
Nathan Binkert
5e9bc06457 Make it so we append jobs to the joblist in the for loop not
outside of the loop so we get all of the jobs, not just the
last one.

util/pbs/send.py:
    fix indent

--HG--
extra : convert_revision : eee9546b4945ff949fdfdf339fc95a23603b47d3
2005-02-14 20:22:27 -05:00
Ali Saidi
b1ab7b53c3 output dir changes to python files
util/pbs/job.py:
    pass output dir to m5 directly

--HG--
extra : convert_revision : 00d1568bb2da3b3e646fc75b4884314bf4cb2d71
2005-02-14 18:54:38 -05:00
Nathan Binkert
200246d1ca Some cosmetic changes to MyPOpen
util/pbs/pbs.py:
    More tweaks that I forgot

--HG--
extra : convert_revision : 7298f91b80bc7d8d946be93fc622e5f9f6e155f9
2005-02-10 00:02:51 -05:00
Nathan Binkert
8efd7d9063 More fixes to the pbs stuff to make it more robust.
sim/pyconfig/SConscript:
    Embed the jobfile.py script into the binary so that we don't
    need to copy it into the Base directory every time.
test/genini.py:
    Add the util/pbs directory to the path so we can get to
    jobfile.py
    Add a -I argument to set to add to the path.
util/pbs/pbs.py:
    Create a MyPOpen class.  This is a lot like the popen2.Popen3 class
    in the python library except that my version allows redirection of
    standard in and standard out to a file instead of a pipe.
    Use this popen class to execute qsub or ssh qsub.  This was important
    for the ssh version of qsub because we need to pipe the script into
    standard in of ssh so that the script can get to the qsub command.
    (Otherwise we have a problem discovering the path.)
util/pbs/send.py:
    Tweak the script so it figures out paths in NFS correctly.
    Use the new system for running qsub.

--HG--
extra : convert_revision : 1289915ba99cec6fd464b71215c32d2197ff2824
2005-02-09 23:55:21 -05:00
Nathan Binkert
7fed053beb More fixes for running from anywhere.
util/pbs/send.py:
    always access the job directory via full path

--HG--
extra : convert_revision : 1792aadb39428e7c91953ac58f6da212b7f92835
2005-02-09 18:12:39 -05:00
Nathan Binkert
061f40df08 Fixes to thes pbs send script
util/pbs/send.py:
    - add a -d to set the job root directory allowing one to run
    send.py from anywhere.
    - specify full paths to files instead of relative paths to make -d
    work and to allow ssh qsub to work again.
    - make the Link directory only copy links that point to regular files.

--HG--
extra : convert_revision : dd330cee08b97c5d72c3d58ef123f83ac7ccede7
2005-02-09 16:20:53 -05:00
Nathan Binkert
fc64ab8102 Make pbs submission scripts available to all.
Fix up configuration scrupts to have better support for
running on the simulation pool.

--HG--
extra : convert_revision : 0178c8600b193d6c0ca69163fb735a7fa0e70782
2005-02-05 13:49:17 -05:00