XCOFDK is the architecture of an eXtensible, Customizable and Object-oriented Framework Development Kit.
This project presents an implementation of XCOFDK for Python through the package xcofdk.
It provides Python applications with a runtime environment capable of:
- stable and reliable execution at any time,
- running active services responsible for life-cycle management,
- managed, unified error handling,
- parallel or concurrent processing out-of-the-box,
- instant access for transparent task or process communication,
- supporting free-threaded (FT) Python (starting with the stable Python version 3.14.0),
- providing API for auxiliary, bundled services commonly expected for application developement.
XCOFDK is available for Python versions 3.8+ on both POSIX and Windows platfroms.
NOTE:
- By installing you agree to the terms and conditions of use of the software (see section Licensing below).
Install using PyPI package xcofdk:
$> python3 -m pip install xcofdkIn general, the runtime environment RTE of the framework of XCOFDK for Python is responsible for the complete execution (of part) of the task model of a program which uses the multithreading subsystem of XCOFDK to construct that (part of the) task model. As such and except for the traditional 'Hello world!', any possible quick-start and sample porgram is by definition more than a 2-liner, especially when demonstrating some major aspects in practice.
Hence, the console program quickStart.py demonstrates (partly advanced) use of the framework illustrating many of framework's features available for applications developed for multitasking. The task model of quickStart.py is composed of task instances designed for rapid construction (RC) by passing a (regular) callback function to the contructor of the respective class:
- a message-driven server task, an instance of class MessageDrivenTask,
- a few asynchronous client tasks, instances of class AsyncTask,
- an (a)synchronous starter (or main) task, an instance of class AsyncTask or SyncTask.
Note that the above-mentioned RC task classes are each an implementation of the interface class IRCTask or IRCCommTask, respectively.
The application is designed to provide the execution of a given algorithm, here CartProdAlgo(), several times using both multithreading and multiprocessing. Each time an execution is requested, the algorithm constructs a random Cartesian Product (CP) and properly returns the result to the requestor, which is either an application (client) task or a child process.
The output of the program is as shown below:
- with the command line option for small CPs supplied:
$> python3 -m quickStart --small-cartesian-product ... [14:12:13.121 XINF][MainThread] Welcome to XCOFDK in TerminalMode of RTE. [14:12:13.124 XINF][Tsk_501002] Running async. MainTask, current host thread: Tsk_501002 [14:12:13.129 XINF][Tsk_501002] Starting 5x child processes... [14:12:13.137 XINF][CTsk_501001] Task AlgoSrv received first delivered message. [14:12:13.178 XINF][Tsk_501002] Starting 5x async. algo-client tasks... [14:12:14.841 XINF][CTsk_501001] Put request to stop the framework. [14:12:15.087 XINF] Got total of 29x small CartProdAlgo executions: [14:12:13.144][TID:501001:AlgoSrv] digitSet='439811' size=7776 tail: 11113 , 11119 , 11118 , 11111 , 11111 [14:12:13.144][TID:501002] digitSet='220997' size=7776 tail: 77772 , 77770 , 77779 , 77779 , 77777 [14:12:13.187][TID:501003] digitSet='268683' size=7776 tail: 33336 , 33338 , 33336 , 33338 , 33333 [14:12:13.190][TID:501004:AlgoC_4] digitSet='735711' size=7776 tail: 11113 , 11115 , 11117 , 11111 , 11111 [14:12:13.203][TID:501006:AlgoC_6] digitSet='865122' size=7776 tail: 22226 , 22225 , 22221 , 22222 , 22222 [14:12:13.204][TID:501003:AlgoC_3] digitSet='539839' size=7776 tail: 99993 , 99999 , 99998 , 99993 , 99999 [14:12:13.209][TID:501005] digitSet='470867' size=7776 tail: 77777 , 77770 , 77778 , 77776 , 77777 [14:12:13.231][TID:501007] digitSet='962670' size=7776 tail: 00006 , 00002 , 00006 , 00007 , 00000 [14:12:13.232][TID:501003] digitSet='818163' size=7776 tail: 33331 , 33338 , 33331 , 33336 , 33333 [14:12:13.247][TID:501003:AlgoC_3] digitSet='242801' size=7776 tail: 11114 , 11112 , 11118 , 11110 , 11111 [14:12:13.254][TID:501004] digitSet='657175' size=7776 tail: 55555 , 55557 , 55551 , 55557 , 55555 [14:12:13.258][TID:501005:AlgoC_5] digitSet='944577' size=7776 tail: 77774 , 77774 , 77775 , 77777 , 77777 [14:12:13.273][TID:501004:AlgoC_4] digitSet='524020' size=7776 tail: 00002 , 00004 , 00000 , 00002 , 00000 [14:12:13.296][TID:501006] digitSet='022271' size=7776 tail: 11112 , 11112 , 11112 , 11117 , 11111 [14:12:13.312][TID:501007:AlgoC_7] digitSet='868329' size=7776 tail: 99996 , 99998 , 99993 , 99992 , 99999 [14:12:13.318][TID:501004] digitSet='559993' size=7776 tail: 33335 , 33339 , 33339 , 33339 , 33333 [14:12:13.340][TID:501005] digitSet='557954' size=7776 tail: 44445 , 44447 , 44449 , 44445 , 44444 [14:12:13.367][TID:501006:AlgoC_6] digitSet='183362' size=7776 tail: 22228 , 22223 , 22223 , 22226 , 22222 [14:12:13.381][TID:501005:AlgoC_5] digitSet='038110' size=7776 tail: 00003 , 00008 , 00001 , 00001 , 00000 [14:12:13.422][TID:501007] digitSet='609383' size=7776 tail: 33330 , 33339 , 33333 , 33338 , 33333 [14:12:13.465][TID:501006] digitSet='149734' size=7776 tail: 44444 , 44449 , 44447 , 44443 , 44444 [14:12:13.515][TID:501007:AlgoC_7] digitSet='174329' size=7776 tail: 99997 , 99994 , 99993 , 99992 , 99999 [14:12:13.731][TID:501002] digitSet='013871' size=7776 tail: 11111 , 11113 , 11118 , 11117 , 11111 [14:12:13.732][TID:501001:AlgoSrv] digitSet='756316' size=7776 tail: 66665 , 66666 , 66663 , 66661 , 66666 [14:12:14.703][PID:18098] digitSet='16217270' size=2097152 tail: 0000001 , 0000007 , 0000002 , 0000007 , 0000000 [14:12:14.758][PID:18099] digitSet='68090537' size=2097152 tail: 7777779 , 7777770 , 7777775 , 7777773 , 7777777 [14:12:14.762][PID:18097] digitSet='99538187' size=2097152 tail: 7777773 , 7777778 , 7777771 , 7777778 , 7777777 [14:12:14.771][PID:18100] digitSet='06201049' size=2097152 tail: 9999990 , 9999991 , 9999990 , 9999994 , 9999999 [14:12:14.772][PID:18101] digitSet='58426844' size=2097152 tail: 4444442 , 4444446 , 4444448 , 4444444 , 4444444 [14:12:15.087 XINF] Done, elapsed time for small CartProdAlgo: 0:00:02.041763
- lines with [TID:nnnn] inside are each the CP result of a request made by or on behalf of an application (client) task,
- lines with [PID:nnnn] inside are each the CP result of a request made out of a child process.
The module function Main() represents program's entry point which is basically organized in accordance to the common pattern of use of the framework. When called, it performs below activities:
- step 1: configuration of framework's RTE
depending on the supplied command line options (if any):
- enable RTE policy for TerminalMode,
- bypass RTE policy for experimental free-threading guard,
- step 2: start of the framework,
- step 3: create and start application's message-driven server task
which will be responsible for:
- triggering new CP calculations on behalf of application (client) tasks the server receives request messages from,
- managing the TerminalMode (if enabled):
- wait for running child processes (if any) to complete,
- leave the TerminalMode by putting a stop request to the framework,
- step 4: create and start application's starter task,
responsible for:
- start of both child processes and client tasks,
- wait for client tasks to complete,
- request the server task to quit,
- step 5: if the RTE policy TerminalMode is disabled:
- wait for running child processes (if any) to complete,
- step 6: wait for framework's coordinated shutdown:
- if the RTE policy TerminalMode is enabled, then the framework will wait before entering its shutdown sequence
as long as the TerminalMode is not left (see step 3 above), - otherwise it will wait as long as there are application tasks still running (see RTE policy AutoStop),
- if the RTE policy TerminalMode is enabled, then the framework will wait before entering its shutdown sequence
- step 7: collect and put out the CP results.
def Main(cmdLineOpts_ : CLOptions):
_startTime = datetime.now()
# step 1: configure framework's RTE for terminal mode and/or
# experimental free-threaded Python (if enabled via CmdLine)
if cmdLineOpts_.isSmallCartProdEnabled:
DisableBigCartProd()
rtecfg.RtePolicyEnableTerminalMode()
if cmdLineOpts_.isFreeThreadingGuardBypassed:
rtecfg.RtePolicyBypassExperimentalFreeThreadingGuard()
# step 2: start framework
if not fwapi.StartXcoFW(fwStartOptions_=cmdLineOpts_.GetSuppliedFwOptions()):
return 71
# step 3: create and start appplication's algo-server task
_algoSrv = MessageDrivenTask(AlgoServerCBTgt, aliasName_='AlgoSrv', bRefToCurTaskRequired_=True, pollingFrequency_=20)
_algoSrv.Start()
# step 4: create and start appplication's starter task
# (which will create and start both child processes and client tasks, too)
_count, _procPool = 5, []
_starterTsk = CreateStartStarterTask(cmdLineOpts_, _algoSrv.taskUID, _count, _procPool)
# step 5: if not in terminal mode, wait for running child processes to complete
if not rtecfg.RtePolicyGetConfig().isTerminalModeEnabled:
fwapi.JoinProcesses()
# step 6: wait for framework's coordinated shutdown
_bLcErrorFree = fwapi.JoinXcoFW()
# step 7: collect and print out results of CP requests executed
if _bLcErrorFree:
_procPoolRes = [_pp.processSuppliedData for _pp in _procPool if _pp.processSuppliedData is not None]
_cp = _procPoolRes + _starterTsk.GetTaskOwnedData() + _algoSrv.GetTaskOwnedData()
_cp.sort(key=lambda _ee: _ee.cartProdTimestamp)
_cpLEN = len(_cp)
_cp = '\n\t'.join( [str(_ee) for _ee in _cp] )
_msg1 = 'small' if cmdLineOpts_.isSmallCartProdEnabled else 'big'
_msg2 = f'Got total of {_cpLEN}x {_msg1} CartProdAlgo executions:\n\t{_cp}'
xlogif.LogInfo(_msg2)
xlogif.LogInfo(f'Done, elapsed time for {_msg1} CartProdAlgo: ' + str(datetime.now()-_startTime))
# done: check for LC failure (if any)
res = 72 if not _bLcErrorFree else 0
return resMore introductory information relared to XCOFDK for Python available on the wiki pages below:
- 1. Introduction for an overview of the framework,
- 2. Quick Start explaining the above console program in more detail:
- 3. Architecture for the design and subsystems of the framework,
- 4. API Overview for an overview of the API of the framework,
- 6. Basic Examples for introductory examples of
(real-world) programs using the framework.
Note that most of the examples are applications developed using GUI framework tkinter.
XCOFDK is available under MIT License.
| Main page | : | XCOFDK on GitHub |
|---|---|---|
| Wiki | : | XCOFDK Wiki |
| API | : | API Overview |
| Examples | : | xcofdk-py-examples.tar.gz | xcofdk-py-examples.zip |
| Changelog | : | Release highlights & release notes |
| Roadmap | : | Roadmap of XCOFDK |
| License file | : | MIT License |
| Error reporting | : | error-xpy@xcofdk.de |