SonotaCo.JP
SonotaCo Network Japan Forum
SonotaCo.JP Forum Index
homeTop Page  FAQFAQ   検索検索   メンバーリストメンバーリスト   ユーザーグループユーザーグループ   登録する登録する 
 プロフィールプロフィール   プライベートメッセージをチェックするプライベートメッセージをチェックする   ログインログイン 

Some ideas (buffer and detection algorithm)

 
新しいトピックを投稿   トピックに返信    SonotaCo.JP Forum Index -> UFOCapture BBS in English
前のトピックを表示 :: 次のトピックを表示  
投稿者 メッセージ
Lightningwizard



登録日: 2007.10.19
記事: 165

記事日時: Thu Jul 24, 2008 4:45 am    記事の件名: Some ideas (buffer and detection algorithm) 引用付きで返信

After a number of sprite observation sessions I can confirm UFOCapture works indeed much better with Vista SP1 (I still used 2.18 ). No long reinitialization blackouts.

But my laptop has a fairly slow 4200 RPM disk so when writing an uncompressed AVI file the storage may take longer than the time the video spans (e.g. 2 seconds for a 1.2 second movie). The program does not start detecting again until the complete video has been saved, so typically there is a small gap of detection up to a few seconds.

Idea 1)
If possible, the program should ideally keep a buffer in RAM of a certain amount of video frames to ascertain continuity of detection and recording. As I have 2 GB RAM in the system, and 50% is not used, it would be possible to store up to 1500 frames in the buffer (say 20 movies).
If a movie is being recorded and I was doing other things with the harddisk, like playing a previous movie, the resulting file can be incomplete. For this the buffer could be useful. One could even go as far as testing if a written file is complete, if it is not then the file has to be rewritten from the buffer.
Playing a file in UFOCapture could perhaps also be made a little more fluently, e.g. first load into RAM before playing (not straight from disk).

Idea 2)
Currently UFOCapture triggers when the number of pixels that change more than the Detect Level (maximum difference of a pixel in the image) exceed Detect Size.
My Detect Level is set to the noise background, usually 25-35. Yet there are flashes (lightning, but maybe sprites) that are clearly visible over the noise level but only give a few dots (not even inside the brightest part of the flash) marked blue! This means an actual detection failure.
I think the reason is that Detect Level is only the difference between pixel values. If in a black sky there is noise causing 30 levels of difference (15 below the average pixel brightness and 15 above), anything that would be 5 or maybe even 15 values brighter than the average pixel brightness, which would still detected by eye, will not trigger!
Therefore I suggest to use a 'no event' average pixel brightness image as part of the detection criterion. For example, the user should specify the pixel difference level from the 'no event' average brightness.

I hope it makes sense Wink
Oscar

_________________
Weather & Photography
http://www.lightningwizard.com
EuroSprite blog:
http://eurosprite.blogspot.com
トップに戻る
ユーザーのプロフィールを表示  
SonotaCo
Site Admin


登録日: 2004.08.07
記事: 12653
所在地: 139.67E 35.65N

記事日時: Thu Jul 24, 2008 1:36 pm    記事の件名: Re: Some ideas (buffer and detection algorithm) 引用付きで返信

Hi Oscar
Thank you for ideas.
Lightningwizard wrote:
Idea 1)
If possible, the program should ideally keep a buffer in RAM of a certain amount of video frames to ascertain continuity of detection and recording. As I have 2 GB RAM in the system, and 50% is not used, it would be possible to store up to 1500 frames in the buffer (say 20 movies).

This may be possible and surely effective for some users.
Currently, UFOCapture owes the file writing part to Microsoft's DirectShow modules completely.
Once I was planning to write all file writing modules by myself.
There memory buffering and low priority file writing may be possible.
But according to the improvement of Vista, I became thinking that the necessity of it had become small.

Because this is the matter of big development, I cannot decide it now.
I am thinking that there may be next chance when the hi-definition night sky observation becomes possible.
There, probably, I will be forced to develop UFOCaptureV3.


Lightningwizard wrote:
Idea 2)
Therefore I suggest to use a 'no event' average pixel brightness image as part of the detection criterion. For example, the user should specify the pixel difference level from the 'no event' average brightness.


This method was tested during the developing of UFOCapture,but the result was not better than current algorithm.
Also DarkObjMask of current version uses the difference level from time averaged pixel brightness.
I don't know whether it goes well or not but may be you can test similar effect by setting DOlevel 5 and other parameters very sensitive.

Under the restriction of insufficiency S/N of current video cameras, we should give up something.
You may be able to get faint event by setting parameters very sensitive such as
Detect Size 2, DLratio 102, MinDL 10, MinL-N 2. But you will get many noise event also then.
トップに戻る
ユーザーのプロフィールを表示  
Lightningwizard



登録日: 2007.10.19
記事: 165

記事日時: Thu Jul 24, 2008 9:24 pm    記事の件名: 引用付きで返信

Thanks for your answer. I will try the DOlevel - the manual said that setting it too high could cause also brighter objects to become neglected, the opposite of what we wish?

Of course, it is hard to predict the effect of different detection methods with respect to noise, effectiveness of size parameter, etc. I will try to do some simple calculations to understand better the effects. Thinking about it, one could also do a smoothing of the image which results in approximately the average value over time. For example an elve which is a very slight brightness enhancement over the average value over a large area should show up this way. The smoothing would work like a low-pass filter on spatial frequencies, so it would decrease the tail of the Detect Level distribution over an image for the background noise. Not sure if processors are fast enough to smooth in real time a video stream.

Interesting subject Smile
Oscar

_________________
Weather & Photography
http://www.lightningwizard.com
EuroSprite blog:
http://eurosprite.blogspot.com
トップに戻る
ユーザーのプロフィールを表示  
Lightningwizard



登録日: 2007.10.19
記事: 165

記事日時: Fri Jul 25, 2008 12:50 am    記事の件名: 引用付きで返信

Just played with smoothing, and it works very well. Here is my test:

I use images from a video sequence that triggered apparently on just a few pixels, together with noise. The video was recorded with automatic detect level without any margin.
Attached is the green image (series may occur in reverse order), the few orange pixels (hard to see) are the trigger. It is a reflection of lightning on a cloud. The flash itself is easily visible by eye but not detected.

I took frames 24 and 25 into the free program Image Analyzer, the ones just before the trigger and containing the trigger. Attached are both images.
Then I do Combine Images... and subtract image 24 from 25 with and offset of 128. This results in the difference image (attached).

Then one can set a Detect Level threshold, which is the difference between 128, we set it to 158 so this corresponds to Detect Level 30, and we obtain about the same triggering pixels (in white) as UFOCapture, including also some noise.
Attached is also what would be triggered with a Detect level setting of 3.

Now, if we smooth the difference image (strength 30 in Image Analyzer), we can basically set the Detect Level to only 3 without seeing noise! The white area corresponds to the flash. It can even be set to 2 (maybe needs a bit more smoothing before). See attached image.

We can be more succesful if we first de-interlace the frames and compare the differences between odd and even fields, because flashes often have very quick variations. When using interlaced images these variations may cause the differences to become smaller.
In this case we can take image 24, smooth the difference image and get with threshold 3 again a good impression of the flash that triggered it (image attached).

Do we get any noise between frames not containing a trigger? Smoothing the difference frame gives no noise except for thresholds +1 or -1! So we really can use it!


How about a marginally bright sprite?

UFOCapture triggered actually on me moving the camera (I was focussing), and it may have triggered by itself when it reached its maximum (the blue marker appears over the sprite) It should be possible to trigger earlier. The sprite is in the first frame small and dim. Does the smoothing allow detection for this and the later frame?
First, the difference frame clearly shows the sprite and a detect level of 34 allows 15 pixels of the sprite to be visible, with noise starting to show up as well. So, with these settings the sprite would have been detected normally with UFOCapture. However, it would not in case of smoothing with strength of 30. No surprise when the elements are only 3 pixels wide and not bright. In order to show up with level threshold at 3 the smoothing should be around 10. Since sprites and jets often have more vertically oriented lines than horizontally, one could also try smoothing only the vertical. Anyway, the second frame of the sprite would even be detected on smoothing strength 30 and Detect Level 3.

Do not smooth the images themselves before taking the difference, it gives artifacts.
Overall, smoothing works quite well, especially for larger objects but at almost no loss of detection of small objects and smoothing and levels can be tuned for optimal detection.
The biggest question is simply if it can be done in realtime at acceptable computing cost. Perhaps the graphics card could do this instead of the processor?

Oscar

PS. if using Gaussian blur in Paint.NET only radius 5-10 is required for same effect.



M20080721_011630__LWM.jpg
 説明:
orange pixels triggered (they are now green dots after jpg compression...)
 ファイルサイズ:  86.77 KB
 閲覧数:  11590 回

M20080721_011630__LWM.jpg



M20080721_011630__LW_difference.jpg
 説明:
difference in pixel value (negative is smaller than 128, positive larger than 128)
 ファイルサイズ:  96.54 KB
 閲覧数:  11590 回

M20080721_011630__LW_difference.jpg



M20080721_011630__LW_difference_thr30.png
 説明:
difference at Detect Level 30 (not much detected, some noise as well)
 ファイルサイズ:  2.96 KB
 閲覧数:  11590 回

M20080721_011630__LW_difference_thr30.png



M20080721_011630__LW_difference_thr3.png
 説明:
Detect level 3, everything white is trigger
 ファイルサイズ:  79.54 KB
 閲覧数:  11590 回

M20080721_011630__LW_difference_thr3.png



M20080721_011630__LW_difference_smooth_thr3.png
 説明:
Smoothed with strength 30, one can use even Detect Level 3 and get as result a realistic flash detection without much noise.
 ファイルサイズ:  7.39 KB
 閲覧数:  11590 回

M20080721_011630__LW_difference_smooth_thr3.png



M20080721_011630__LW_difference_smooth_thr3_even-odd.png
 説明:
Using the two de-interlaced fields in frame 24 (just before the official trigger), smoothing the difference one can detect at level 3 a flash very well.
 ファイルサイズ:  6.41 KB
 閲覧数:  11590 回

M20080721_011630__LW_difference_smooth_thr3_even-odd.png



z20080701_224653__LW_083s.jpg
 説明:
First frame of the dim sprite. May not have triggered normally because of small size and close to noise level.
 ファイルサイズ:  44.73 KB
 閲覧数:  11590 回

z20080701_224653__LW_083s.jpg



z20080701_224653__LW_084s.jpg
 説明:
Second frame of the dim sprite. Will easily trigger UFOCapture above noise level.
 ファイルサイズ:  44.59 KB
 閲覧数:  11590 回

z20080701_224653__LW_084s.jpg



M20080701_224653__LW_difference_smooth10_thr3_frame1.png
 説明:
At Detect Level 3 this small dim sprite would have been smoothed away at strength 30, but strength 10 is okay for its detection (or use vertical smoothing only)
 ファイルサイズ:  1.79 KB
 閲覧数:  11590 回

M20080701_224653__LW_difference_smooth10_thr3_frame1.png



M20080701_224653__LW_difference_smooth_thr3_frame2.png
 説明:
The brighter luminosity and larger size in the second frame of the sprite cause it to be detected at Detect Level 3 very easily even at smoothing strength 30.
 ファイルサイズ:  1.68 KB
 閲覧数:  11590 回

M20080701_224653__LW_difference_smooth_thr3_frame2.png



_________________
Weather & Photography
http://www.lightningwizard.com
EuroSprite blog:
http://eurosprite.blogspot.com
トップに戻る
ユーザーのプロフィールを表示  
SonotaCo
Site Admin


登録日: 2004.08.07
記事: 12653
所在地: 139.67E 35.65N

記事日時: Fri Jul 25, 2008 12:57 pm    記事の件名: 引用付きで返信

Lightningwizard wrote:
DOlevel - the manual said that setting it too high could cause also brighter objects to become neglected

It is the story when the Detect level and Detect size are used as main threshold.

Lightningwizard wrote:
the opposite of what we wish?

Yes for usual, but No when you set Detect level too sensitive and use DOlevel as the primary threshold.
Then UFOCaptureV2 will detect only pixels that have brightness over time_avarage+DOlevel.
It is unplanned usage but may work. I will test this tonight.

Lightningwizard wrote:
The smoothing would work like a low-pass filter on spatial frequencies, so it would decrease the tail of the Detect Level distribution over an image for the background noise.

Yes. Ancient version of UFOCapture had this feature.
Truly it decreased the noise and increased S/N as expected. But ....
Lightningwizard wrote:
Not sure if processors are fast enough to smooth in real time a video stream.

This was the problem.
Large area smoothing was impossible then and I tested only average of 4 pixels(means 0.5 pixel radius).
The effect was limited and it was progressively replaced by the scintillation mask.

I don't know how large smoothing is possible by current processors.
There are both people who wants higher performance and who wants to operate on poor machines.
It is headaching.
Anyway, I agree that spatial smoothing has possibility to improve the detection ability.

Lightningwizard wrote:
Just played with smoothing, and it works very well. Here is my test:

Oh! amazing Shocked

Lightningwizard wrote:
Now, if we smooth the difference image (strength 30 in Image Analyzer), we can basically set the Detect Level to only 3 without seeing noise!

Oh! smoothing after taking the difference Exclamation I have not tested it.
It is very interesting... may be a new good method.

Lightningwizard wrote:
We can be more successful if we first de-interlace the frames and compare the differences between odd and even fields, because flashes often have very quick variations. When using interlaced images these variations may cause the differences to become smaller.

I am not sure about this. This might miss the far sprites without flashes.

All figures looked reasonable.
I became to think this is worth to test.
I don't know when I can do it. Of course there may be many difficulties, but it will be exciting Cool .

Thanks
トップに戻る
ユーザーのプロフィールを表示  
Lightningwizard



登録日: 2007.10.19
記事: 165

記事日時: Sun Jul 27, 2008 3:30 am    記事の件名: 引用付きで返信

SonotaCo wrote:
Yes for usual, but No when you set Detect level too sensitive and use DOlevel as the primary threshold. Then UFOCaptureV2 will detect only pixels that have brightness over time_avarage+DOlevel. It is unplanned usage but may work. I will test this tonight.

I'm curious about your findings. I did a test last night but there was no thunderstorm nearby (or at least, clouds obscured it). I had set Detect Level at 0, Size at 4-15 and DOLevel to...28! I am not sure if it detects more easily very weak flashes (I hope to know more after Monday) but I did get meteors. The blue marker indicates a similar degree of noise pixels and only a part of the meteor trails, not their beginning even though visible in the peakhold image. This suggests it is not more sensitive than when using normal settings... Going to DOLevel 27 it triggers on noise too often.

SonotaCo wrote:
I don't know how large smoothing is possible by current processors.
There are both people who wants higher performance and who wants to operate on poor machines. It is headaching. Anyway, I agree that spatial smoothing has possibility to improve the detection ability.

My Core2Duo 1.8 GHz runs UFOCapture at 30% total CPU when scintillation mask is on and audio included (I record AM radio sferics). You can keep everybody your friend if you offer smoothing as option and its level adjustable. Smoothing could be done in several ways, there may be more and less efficient algorithms and also smarter things like noise reduction. But since they are filters, could they perhaps be performed by the GPU? De-interlacing perhaps as well?
I have very little knowledge of DirectShow programming.

SonotaCo wrote:
Oh! smoothing after taking the difference Exclamation I have not tested it.
It is very interesting... may be a new good method.

Yes, try displaying the histogram of the difference image, there are very long tails. After smoothing they are gone. I think it requires the same computing power though. If faster, you could also try to first resize the difference image to 25% by some algorithm that doesn't pixelize or produce banding, smooth it with a smaller radius and apply the threshold. If you de-interlace first you have twice more images to analyze of half the size (if it is simply removal of one field without resizing)...

SonotaCo wrote:
I am not sure about this. This might miss the far sprites without flashes.

Sprites often occur in just one field, the other field which is interleaved with it remains dark. If you keep them mixed, the contrast with the background after smoothing will be lower. On the other hand, single fields appear to have more noise.

SonotaCo wrote:
I became to think this is worth to test. I don't know when I can do it. Of course there may be many difficulties, but it will be exciting Cool .

Thanks a lot! Let me know if you have results from the testing Smile

Oscar

_________________
Weather & Photography
http://www.lightningwizard.com
EuroSprite blog:
http://eurosprite.blogspot.com
トップに戻る
ユーザーのプロフィールを表示  
SonotaCo
Site Admin


登録日: 2004.08.07
記事: 12653
所在地: 139.67E 35.65N

記事日時: Sun Jul 27, 2008 10:42 am    記事の件名: 引用付きで返信

I also tested DOlevel main thresholding for two nights.
It worked, but as you are saying, it was not more sensitive than usual methods.
It was almost equal or less.
I remember that I had got same result years ago. There, I gave up this approach.
It seemed the noise of latest frame is dominating the sensitivity.

By the way,
Quote:
Size at 4-15
Isn't it rather large? I am using Size 2 or 3 always.
Quote:
My Core2Duo 1.8 GHz runs UFOCapture at 30% total CPU
But many of the users runs two or more instances simultaneously in one PC.
Quote:
You can keep everybody your friend if you offer smoothing as option and its level adjustable.
Yes, that was the way I had been taking. But, as the result of it, there are too many options.
It became too complex for most of the users Rolling Eyes . That is one of the problem now.

Well.....I may need meditation Confused
トップに戻る
ユーザーのプロフィールを表示  
Lightningwizard



登録日: 2007.10.19
記事: 165

記事日時: Sat Aug 02, 2008 4:32 am    記事の件名: 引用付きで返信

I know of one other way to detect events. Our EuroSprite camera systems at Pic du Midi and Corsica trigger when thresholds on row and column sums are exceeded, for a given number of rows and columns. I'm not sure how well it could potentially work, because I have captured sprites that were missed by Pic du Midi (but also the reverse sometimes)... but I don't know its threshold settings. The advantage may be that per row and column the noise should yield a reasonably steady sum, and the computing cost is low. The disadvantage is that small dim objects like most meteors are not detected because of their small contribution to the sum, and if bright steady objects are in view, the sum of the corresponding row or column sum changes relatively even less for such meteor (if measured in percentage, not absolute value)
So, smoothing may still work best, if it would run efficiently, because small scale objects can be detected independently from what occurs in the rest of the image.

For a future version of UFOCapture you could provide a basic mode with settings like "meteor", "sprite", etc, and an advanced mode with the full range of detection settings for the ones who want to tweak.

Oscar

_________________
Weather & Photography
http://www.lightningwizard.com
EuroSprite blog:
http://eurosprite.blogspot.com
トップに戻る
ユーザーのプロフィールを表示  
Lightningwizard



登録日: 2007.10.19
記事: 165

記事日時: Sat Aug 16, 2008 7:11 am    記事の件名: 引用付きで返信

Some other ideas yet.
1) Usually when an object moves or appears very shortly, it will make differences between even and odd rows of an interlaced frame. One could calculate the sum of pixel values over the even rows and over the odd rows, which per image should be pretty much the same (difference near 0), unless an event happens. Advantage is its fast calculation and resistance to noise. Disadvantage: may not detect the smaller or slower events. But it may work in combination with the current method.

2) One can take the difference values between two frames (or between pixels of new frame and an average image) and calculate the center of 'mass'. If it is noise it would likely be very close to the absolute center of the image. It should be rare to find this result for a real object. It could be done for several sectors of the image separately, perhaps with some overlap. It could also be used in addition to the existing method, for the range of 0-DetectLevel (for weak events between the noise).

Oscar

_________________
Weather & Photography
http://www.lightningwizard.com
EuroSprite blog:
http://eurosprite.blogspot.com
トップに戻る
ユーザーのプロフィールを表示  
Lightningwizard



登録日: 2007.10.19
記事: 165

記事日時: Tue Dec 09, 2008 3:27 am    記事の件名: 引用付きで返信

Dear SonotaCo,

I hope you don't mind I propose yet another idea...

Instead of the two bars for size and detect level, display a live histogram of the number of pixels that have (absolute) brightness differences between two frames. Noise will show as a peak at zero in the middle (or at the left side if absolute or positive-only is used), decreasing towards the right.

In the standard no-event video, the histogram will remain nearly constant. If a luminous event occurs, the middle value bars will increase briefly. You can compare this to a equalizer on an audio system.
This allows a very similar treatment as UFOCapture currently does with automatic detect level: monitor the average size of the bars of the histogram, set the trigger level for each bar above the noise level, and trigger when any of the bars exceeds the threshold. The largest bars will be more constant than the current Detect Level bar.

Now, the user can choose which range of bars to use. Using the bars closer to zero allows to detect events (like elves) which are weaker than typical maximum noise fluctuations the current UFOCapture is able to detect. Excluding the brightest bars could help against triggering by bright airplane lights and electric noises. A sprite or meteor consisting of bright and dim parts will still raise a trigger on the less bright parts in that case. This approach allows the size criterion to be different according to the brightness.

The advantage of this method is that it remains a similar load for the processor, it is not limited to the difference level of the most noisy pixel, and it offers flexibility for the type of objects to detect or avoid.

What do you think?
Oscar

_________________
Weather & Photography
http://www.lightningwizard.com
EuroSprite blog:
http://eurosprite.blogspot.com
トップに戻る
ユーザーのプロフィールを表示  
SonotaCo
Site Admin


登録日: 2004.08.07
記事: 12653
所在地: 139.67E 35.65N

記事日時: Tue Dec 09, 2008 9:26 am    記事の件名: 引用付きで返信

Hi Oscar
Thank you for nice suggestion.

like a equalizer on an audio system.... Shocked
multiple thresholds on size....
use mode value instead of mean value as the noise level ....

Your idea is quite attractive every time Very Happy .

Well, I will consider about this if I can have a chance to make a professional version.
To say the truth, I am waiting a new framework that uses high definition technology.
When it becomes common for night sky observer, I will do it.

SonotaCo
トップに戻る
ユーザーのプロフィールを表示  
特定期間内の記事を表示:   
新しいトピックを投稿   トピックに返信    SonotaCo.JP Forum Index -> UFOCapture BBS in English All times are GMT + 9 Hours
Page 1 of 1

 
移動先:  
新規投稿: 不可
返信投稿: 不可
記事編集: 不可
記事削除: 不可
投票参加: 不可
このフォーラムで添付ファイルを投稿 できません
このフォーラムでファイルをダウンロード できます


Powered by phpBB © 2001, 2005 phpBB Group
Copyright ©2004 SonotaCo Network. All Rights Reserved.