An exception was thrown by a TaskScheduler
An exception was thrown by a TaskScheduler
Hi Bernd,
This night our system stopped, the following errors I captured. The Overlay Player was loaded but completely freezed. Even the next scripts didn't start, after 02.00 it was quiet. I see as a first error "An exception was thrown by a TaskScheduler" but I can't bring that error back to a recently change of our scripts / programming.
Overlaylog
Debug-Log:
This night our system stopped, the following errors I captured. The Overlay Player was loaded but completely freezed. Even the next scripts didn't start, after 02.00 it was quiet. I see as a first error "An exception was thrown by a TaskScheduler" but I can't bring that error back to a recently change of our scripts / programming.
Overlaylog
Code: Select all
28-5-2014 2:56:33: Overlay:Commercials, Command:Advertising (Hele_uur), Start:28-5-2014 3:00:00 processed
28-5-2014 2:56:44: WARNING: Overlay-Scheduler-Exception:'An exception was thrown by a TaskScheduler.'!
28-5-2014 3:56:39: WARNING: Overlay 'Commercials@2014-05-28T04:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=3).
28-5-2014 3:56:39: WARNING: Overlay 'Nieuws@2014-05-28T04:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=3).
28-5-2014 3:56:39: WARNING: Overlay 'Uuropener@2014-05-28T04:00:01' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=3).
28-5-2014 4:56:40: WARNING: Overlay 'Commercials@2014-05-28T05:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=6).
28-5-2014 4:56:40: WARNING: Overlay 'Nieuws@2014-05-28T05:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=6).
28-5-2014 4:56:40: WARNING: Overlay 'Uuropener@2014-05-28T05:00:01' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=6).
28-5-2014 5:56:42: WARNING: Overlay 'Commercials@2014-05-28T06:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=9).
28-5-2014 5:56:42: WARNING: Overlay 'Nieuws@2014-05-28T06:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=9).
28-5-2014 5:56:42: WARNING: Overlay 'Uuropener@2014-05-28T06:00:01' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=9).
28-5-2014 6:56:43: WARNING: Overlay 'Commercials@2014-05-28T07:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=12).
28-5-2014 6:56:43: WARNING: Overlay 'Nieuws@2014-05-28T07:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=12).
28-5-2014 6:56:43: WARNING: Overlay 'Uuropener@2014-05-28T07:00:01' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=12).
28-5-2014 7:56:45: WARNING: Overlay 'Commercials@2014-05-28T08:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=15).
28-5-2014 7:56:45: WARNING: Overlay 'Nieuws@2014-05-28T08:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=15).
28-5-2014 7:56:45: WARNING: Overlay 'Uuropener@2014-05-28T08:00:01' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=15).
28-5-2014 7:58:34: Calendar 'D:\_Reclame\Overlay.calendar' updated (last modified: 22-5-2014 23:03:43)
Code: Select all
8-5-2014 1:58:46: Processing Script-Line 3 (Retries=60): 2 Random : REMOTE:192.168.1.13:8088//Various Dance Classics Pop Edition (Filter=, Options=SuppressHistoryCheck)
28-5-2014 1:58:46: GetNextTracks processed, 3 script result entries returned.
28-5-2014 2:00:00: PLS-PlayNext.
28-5-2014 2:56:33: Overlay 'Commercials@2014-05-28T03:00:00' starts processing.
28-5-2014 2:56:33: Overlay 'Commercials@2014-05-28T03:00:00' start loading.
28-5-2014 2:56:33: Overlay 'Commercials@2014-05-28T03:00:00' ProgressChanged State=0.
28-5-2014 2:56:33: OverlayScheduler: Overlay:Commercials, Command:Advertising (Hele_uur), Start:28-5-2014 3:00:00 processed
28-5-2014 2:56:33: Overlay 'Commercials@2014-05-28T03:00:00' starting LoadAdvertSlot.
28-5-2014 2:56:33: Overlay 'Commercials@2014-05-28T03:00:00' LoadAdvertSlot 2 Advert-Tracks (4 Total) found.
28-5-2014 2:56:33: Overlay 'Commercials@2014-05-28T03:00:00' LoadAdvertSlot finished 4 Total-Tracks (00:01:17.0640000 Playtime).
28-5-2014 2:56:33: Overlay 'Commercials@2014-05-28T03:00:00' finish loading.
28-5-2014 2:56:33: Overlay 'Commercials@2014-05-28T03:00:00' starting (4 Tracks).
28-5-2014 2:56:44: Overlay 'Nieuws@2014-05-28T03:00:00' enqueued as parallel overlay.
28-5-2014 2:56:44: OverlayScheduler: WARNING: Overlay-Scheduler-Exception:'An exception was thrown by a TaskScheduler.'!
28-5-2014 2:56:45: Overlay-Scheduler-Exception: 28-5-2014 2:56:44 [v3.0.13.24]:********** Error/Exception Information **********
Error Message: An exception was thrown by a TaskScheduler.
Error Type: System.Threading.Tasks.TaskSchedulerException
Error Source: mscorlib.Void ScheduleAndStart(Boolean)
Inner Exception: Exception of type 'System.OutOfMemoryException' was thrown.
********** System Stack Trace **********
at System.Threading.Tasks.Task.ScheduleAndStart(Boolean needsProtection)
at System.Threading.Tasks.Task.InternalStartNew(Task creatingTask, Delegate action, Object state, CancellationToken cancellationToken, TaskScheduler scheduler, TaskCreationOptions options, InternalTaskOptions internalOptions, StackCrawlMark& stackMark)
at System.Threading.Tasks.TaskFactory.StartNew(Action action, TaskCreationOptions creationOptions)
at ProppFrexx.Player.b.h(j A_0)
at ProppFrexx.da.a(j A_0)
at ProppFrexx.da.a(Object A_0, ReminderEventArgs A_1)
Error Message: Exception of type 'System.OutOfMemoryException' was thrown.
Error Type: System.OutOfMemoryException
Error Source: mscorlib.Void StartInternal(System.Security.Principal.IPrincipal, System.Threading.StackCrawlMark ByRef)
********** System Stack Trace **********
at System.Threading.Thread.StartInternal(IPrincipal principal, StackCrawlMark& stackMark)
at System.Threading.Thread.Start(StackCrawlMark& stackMark)
at System.Threading.Thread.Start(Object parameter)
at System.Threading.Tasks.ThreadPoolTaskScheduler.QueueTask(Task task)
at System.Threading.Tasks.Task.ScheduleAndStart(Boolean needsProtection)
********** Environment Stack Trace **********
at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo)
at System.Environment.get_StackTrace()
at ProppFrexx.ab.a(Exception A_0, Boolean A_1)
at ProppFrexx.da.a(Object A_0, ReminderEventArgs A_1)
at DevExpress.XtraScheduler.SchedulerStorageBase.RaiseReminderAlert(ReminderEventArgs e)
at DevExpress.XtraScheduler.SchedulerStorageBase.OnRemindersAlerted(Object sender, ReminderEventArgs e)
at DevExpress.XtraScheduler.Native.ReminderEngine.OnReminderAlert(Object sender, ReminderBaseEventArgs e)
at DevExpress.XtraScheduler.Native.ReminderEngineBase.RaiseReminderAlert(ReminderBaseEventArgs e)
at DevExpress.XtraScheduler.Native.ReminderEngineBase.RaiseRemindersAlertsCore(ReminderBaseEventArgs args)
at DevExpress.XtraScheduler.Native.ReminderEngineBase.RaiseRemindersAlerts(DateTime currentTime)
at DevExpress.XtraScheduler.Native.ReminderEngineBase.OnTimerTick(Object sender, EventArgs e)
at DevExpress.XtraScheduler.Native.LeakSafeTimerTickEventRouter.OnTimerTick(Object sender, EventArgs e)
at System.Windows.Forms.Timer.OnTick(EventArgs e)
at System.Windows.Forms.Timer.TimerNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
at System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG& msg)
at System.Windows.Forms.Application.ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(IntPtr dwComponentID, Int32 reason, Int32 pvLoopData)
at System.Windows.Forms.Application.ThreadContext.RunMessageLoopInner(Int32 reason, ApplicationContext context)
at System.Windows.Forms.Application.ThreadContext.RunMessageLoop(Int32 reason, ApplicationContext context)
at System.Windows.Forms.Application.Run(Form mainForm)
at ProppFrexx.dw.a(String[] A_0)
********** End **********
28-5-2014 2:56:45: Overlay 'Uuropener@2014-05-28T03:00:01' enqueued as parallel overlay.
28-5-2014 2:58:45: ProgramScheduler: Program:Muziekmix (Muziekmix), Start:28-5-2014 3:00:00, End:28-5-2014 4:00:00, Started:True
28-5-2014 2:58:46: Processing Script-Line 1 (Retries=60): 2 Random : REMOTE:192.168.1.13:8088//7-12-inch singles Tipparade (Filter=, Options=SuppressHistoryCheck)
28-5-2014 2:58:46: GetNextTracks processed, 2 script result entries returned.
28-5-2014 2:58:46: Processing Script-Line 2 (Retries=60): 2 Random : REMOTE:192.168.1.13:8088//7-12-inch singles Top 40 (Filter=, Options=SuppressHistoryCheck)
28-5-2014 2:58:46: GetNextTracks processed, 3 script result entries returned.
28-5-2014 2:58:46: Processing Script-Line 3 (Retries=60): 2 Random : REMOTE:192.168.1.13:8088//Various Dance Classics Pop Edition (Filter=, Options=SuppressHistoryCheck)
28-5-2014 2:58:46: GetNextTracks processed, 2 script result entries returned.
28-5-2014 2:58:46: Processing Script-Line 4 (Retries=60): 2 Random : REMOTE:192.168.1.13:8088//Various Top 40 (Filter=, Options=SuppressHistoryCheck)
28-5-2014 2:58:46: GetNextTracks processed, 3 script result entries returned.
28-5-2014 2:58:46: Processing Script-Line 5 (Retries=60): 2 Random : REMOTE:192.168.1.13:8088//Various Hitzone (Filter=, Options=SuppressHistoryCheck)
28-5-2014 2:58:46: GetNextTracks processed, 3 script result entries returned.
28-5-2014 2:58:46: Processing Script-Line 1 (Retries=60): 2 Random : REMOTE:192.168.1.13:8088//7-12-inch singles Tipparade (Filter=, Options=SuppressHistoryCheck)
28-5-2014 2:58:46: GetNextTracks processed, 3 script result entries returned.
28-5-2014 2:58:46: Processing Script-Line 2 (Retries=60): 2 Random : REMOTE:192.168.1.13:8088//7-12-inch singles Top 40 (Filter=, Options=SuppressHistoryCheck)
28-5-2014 2:58:46: GetNextTracks processed, 3 script result entries returned.
28-5-2014 2:58:48: DJPlayerReport: Player=A, Event=TrackLoad: Brennan Heart And Jonathan Mendelsohn - Imaginary
28-5-2014 3:56:39: OverlayScheduler: WARNING: Overlay 'Commercials@2014-05-28T04:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=3).
28-5-2014 3:56:39: OverlayScheduler: WARNING: Overlay 'Nieuws@2014-05-28T04:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=3).
28-5-2014 3:56:39: OverlayScheduler: WARNING: Overlay 'Uuropener@2014-05-28T04:00:01' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=3).
28-5-2014 3:58:45: ProgramScheduler: WARNING: Program 'Muziekmix @ 28-5-2014 4:00:00' could not be started (another program is already starting at the same time)!
28-5-2014 4:56:40: OverlayScheduler: WARNING: Overlay 'Commercials@2014-05-28T05:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=6).
28-5-2014 4:56:40: OverlayScheduler: WARNING: Overlay 'Nieuws@2014-05-28T05:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=6).
28-5-2014 4:56:40: OverlayScheduler: WARNING: Overlay 'Uuropener@2014-05-28T05:00:01' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=6).
28-5-2014 4:58:46: ProgramScheduler: WARNING: Program 'Piratenmix @ 28-5-2014 5:00:00' could not be started (another program is already starting at the same time)!
28-5-2014 5:56:42: OverlayScheduler: WARNING: Overlay 'Commercials@2014-05-28T06:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=9).
28-5-2014 5:56:42: OverlayScheduler: WARNING: Overlay 'Nieuws@2014-05-28T06:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=9).
28-5-2014 5:56:42: OverlayScheduler: WARNING: Overlay 'Uuropener@2014-05-28T06:00:01' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=9).
28-5-2014 5:58:46: ProgramScheduler: WARNING: Program 'Piratenmix @ 28-5-2014 6:00:00' could not be started (another program is already starting at the same time)!
28-5-2014 6:56:43: OverlayScheduler: WARNING: Overlay 'Commercials@2014-05-28T07:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=12).
28-5-2014 6:56:43: OverlayScheduler: WARNING: Overlay 'Nieuws@2014-05-28T07:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=12).
28-5-2014 6:56:43: OverlayScheduler: WARNING: Overlay 'Uuropener@2014-05-28T07:00:01' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=12).
28-5-2014 6:58:47: ProgramScheduler: WARNING: Program 'Muziekmix @ 28-5-2014 7:00:00' could not be started (another program is already starting at the same time)!
28-5-2014 7:56:45: OverlayScheduler: WARNING: Overlay 'Commercials@2014-05-28T08:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=15).
28-5-2014 7:56:45: OverlayScheduler: WARNING: Overlay 'Nieuws@2014-05-28T08:00:00' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=15).
28-5-2014 7:56:45: OverlayScheduler: WARNING: Overlay 'Uuropener@2014-05-28T08:00:01' not processed Current 'Commercials@2014-05-28T03:00:00' still running (Status (Shown) background thread still busy, Count=15).
Radio Aa en Hunze - Twaalf hunebedden één geluid
www.radioaaenhunze.nl - Phone studio +31 (0)592 70 30 30
www.radioaaenhunze.nl - Phone studio +31 (0)592 70 30 30
Re: An exception was thrown by a TaskScheduler
Hi Bernd,
Memory ... but how? On the system runs only PF en Media Libary. Everything is stored on internal harddisks. Out of memory, in normal state the system uses only 10% of a total of 16GB of memory. The only possibility if ProppFrexx is stable, 2 days it kills himself through a memory failure. Everyday a reboot? How can we fix this, you know how the system is configured by us (Forum user XTended is always talking about the same system) we are using Media Server, System is up to date, Everything is stored on internal drives, 16GB of memory.
MediaLibary has:
Loaded Media Libraries: 91
Total songs / commercials / jingles: +/- 250.000
Is that to much for the system? We are at this moment out of options, we don't know how to get it stable.
Memory ... but how? On the system runs only PF en Media Libary. Everything is stored on internal harddisks. Out of memory, in normal state the system uses only 10% of a total of 16GB of memory. The only possibility if ProppFrexx is stable, 2 days it kills himself through a memory failure. Everyday a reboot? How can we fix this, you know how the system is configured by us (Forum user XTended is always talking about the same system) we are using Media Server, System is up to date, Everything is stored on internal drives, 16GB of memory.
MediaLibary has:
Loaded Media Libraries: 91
Total songs / commercials / jingles: +/- 250.000
Is that to much for the system? We are at this moment out of options, we don't know how to get it stable.
Radio Aa en Hunze - Twaalf hunebedden één geluid
www.radioaaenhunze.nl - Phone studio +31 (0)592 70 30 30
www.radioaaenhunze.nl - Phone studio +31 (0)592 70 30 30
Re: An exception was thrown by a TaskScheduler
Dear Bernd,
We use the Media Library Server since February 2014...
All ideas you gave us, we use. So it must be something else.
We use the Media Library Server since February 2014...
All ideas you gave us, we use. So it must be something else.
Re: An exception was thrown by a TaskScheduler
Have you may be still added/used your media libs within PF - plus the remote libs?
What is your memory utilization of ProppFrexx within the TaskManager?
When you are only using remote libs and no additional media libs within PF the memory consumption should be at max around 500 MB!
What is yours...
I mean an out-of-memory except is an OS related exception and if it comes, it means out of memory...
Else can we do a TeamViewer session to take a look at?
What is your memory utilization of ProppFrexx within the TaskManager?
When you are only using remote libs and no additional media libs within PF the memory consumption should be at max around 500 MB!
What is yours...
I mean an out-of-memory except is an OS related exception and if it comes, it means out of memory...
Else can we do a TeamViewer session to take a look at?
Bernd - radio42
ProppFrexx ONAIR - The Playout and Broadcast Automation Solution
ProppFrexx ONAIR - The Playout and Broadcast Automation Solution
Re: An exception was thrown by a TaskScheduler
We only use the Media Library Server, so there aren't any loaded music files anymore within PF; all is set in the MLS.
Our memory consumption looks a little bit higher; see image I created couple minutes ago.
Our memory consumption looks a little bit higher; see image I created couple minutes ago.
- Attachments
-
- TaskManager.jpg (74.18 KiB) Viewed 12635 times
Re: An exception was thrown by a TaskScheduler
As you might see from the logs.
The exception thrown was a 'System.OutOfMemoryException' - which effectively means: out of memory.
There is not much I can do about this kind of error.
If you are using a lot of media libs resp. a lot of entries within your media libs, it might be a good idea to use the Media Library Server.
See here for details: viewtopic.php?f=9&t=1014
The exception thrown was a 'System.OutOfMemoryException' - which effectively means: out of memory.
There is not much I can do about this kind of error.
If you are using a lot of media libs resp. a lot of entries within your media libs, it might be a good idea to use the Media Library Server.
See here for details: viewtopic.php?f=9&t=1014
Bernd - radio42
ProppFrexx ONAIR - The Playout and Broadcast Automation Solution
ProppFrexx ONAIR - The Playout and Broadcast Automation Solution
Re: An exception was thrown by a TaskScheduler
Higher than what?
ProppFrexx OnAir uses less than 400MB!?
So please watch, if that increases over time...
ProppFrexx OnAir uses less than 400MB!?
So please watch, if that increases over time...
Bernd - radio42
ProppFrexx ONAIR - The Playout and Broadcast Automation Solution
ProppFrexx ONAIR - The Playout and Broadcast Automation Solution
Re: An exception was thrown by a TaskScheduler
Note, that ProppFrexx is a 32bit application! So it leverages only up to 3GB of main memory!
Where your local files are stored doesn't relate to main memory.
Normal states (10%) also doesn't really matter, as the peak memory usage is what matters.
Your number of media entries are not too much in general, but might be too much for a 32bit process to hold in memory.
How to Fix?
That's why I suggested to use the Media Library Server! That should be the solution in your case.
Where your local files are stored doesn't relate to main memory.
Normal states (10%) also doesn't really matter, as the peak memory usage is what matters.
Your number of media entries are not too much in general, but might be too much for a 32bit process to hold in memory.
How to Fix?
That's why I suggested to use the Media Library Server! That should be the solution in your case.
Bernd - radio42
ProppFrexx ONAIR - The Playout and Broadcast Automation Solution
ProppFrexx ONAIR - The Playout and Broadcast Automation Solution
Re: An exception was thrown by a TaskScheduler
If you need help to migrate to the Media Library Server let me know...
See the post above...
Basically you define the same media libs as within ProppFrexx now within the MLS. And the you change your regular media libs within PF by Remote Media Lib entries, that's it.
See the post above...
Basically you define the same media libs as within ProppFrexx now within the MLS. And the you change your regular media libs within PF by Remote Media Lib entries, that's it.
Bernd - radio42
ProppFrexx ONAIR - The Playout and Broadcast Automation Solution
ProppFrexx ONAIR - The Playout and Broadcast Automation Solution
Re: An exception was thrown by a TaskScheduler
443.620 + 397.888 = higher than your 500 MB memory consumption...
We'll keep in touch.
Debug is still on
We'll keep in touch.
Debug is still on