Beginner question: making a new kind of device using webcams?

[Navigateur]

I'm a beginner but I'm determined to do this.

How, in a nutshell, do I make a driver that is automatically installed
upon plugging in 2 specific webcams, that when called upon does some
real-time vision-based detection on the images (using OpenCV, my own
code) and then exposes custom-fired events and variables based on what
is found in the detection, which can be accessed system-wide by any
application, but which does not expose the images from the cameras, or
automatically install default drivers for webcams.

In other words, a vision-based user interface.

I'm aware of UVC, and AvStream.

Any ideas for ANY part of this are welcome. I'll take all ideas you
have very seriously.

Many thanks,
N

[Maxim S. Shatskih]

Just read the video stream off the camera (ANY camera) using DirectShow, and do your detection there.
--
Maxim S. Shatskih
Windows DDK MVP
***@storagecraft.com
http://www.storagecraft.com

Post by Navigateur
I'm a beginner but I'm determined to do this.
How, in a nutshell, do I make a driver that is automatically installed
upon plugging in 2 specific webcams, that when called upon does some
real-time vision-based detection on the images (using OpenCV, my own
code) and then exposes custom-fired events and variables based on what
is found in the detection, which can be accessed system-wide by any
application, but which does not expose the images from the cameras, or
automatically install default drivers for webcams.
In other words, a vision-based user interface.
I'm aware of UVC, and AvStream.
Any ideas for ANY part of this are welcome. I'll take all ideas you
have very seriously.
Many thanks,
N

[Navigateur]

What's your recommended overall architecture for this?

How would I make the 2 specific cameras be recognized and registered
as one (specific) device? Could I still use UVC drivers + a custom KS
proxy DirectShow plug-in? For events/variables, should I use
DirectShow events, CORBA or something else? (given that I want the
events/variables to be easily available to multiple applications
running on all platforms, not only by native Windows apps).

I want to make it as simple as possible for end-users to use and for
third-party application developers to develop with. Given this, how
would you design its overall architecture?

On Aug 4, 6:12 am, "Maxim S. Shatskih"

    Just read the video stream off the camera (ANY camera) using DirectShow, and do your detection there.
--
Maxim S. Shatskih
Windows DDK MVP

Post by Navigateur
I'm a beginner but I'm determined to do this.
How, in a nutshell, do I make a driver that is automatically installed
upon plugging in 2 specific webcams, that when called upon does some
real-time vision-based detection on the images (using OpenCV, my own
code) and then exposes custom-fired events and variables based on what
is found in the detection, which can be accessed system-wide by any
application, but which does not expose the images from the cameras, or
automatically install default drivers for webcams.
In other words, a vision-based user interface.
I'm aware of UVC, and AvStream.
Any ideas for ANY part of this are welcome. I'll take all ideas you
have very seriously.
Many thanks,
N

[Maxim S. Shatskih]

Post by Navigateur
How would I make the 2 specific cameras be recognized and registered
as one (specific) device?

You need to design a way to merge the 2 video streams first.

Then, this "merger" should be implemented as one more DirectShow filter.

Post by Navigateur
Could I still use UVC drivers + a custom KS
proxy DirectShow plug-in?

Yes.

Post by Navigateur
DirectShow events, CORBA

How is this related? isn't this a Linux replacement for SOAP and similar things? how is it related to video?

Surely DirectShow supports properties on filters.

[Navigateur]

This is very interesting and enlightening.

Maxim: by events/variables, I mean my own basic detection events/
variables fired by me, not video events. Can DS filters perform bi-
directional inter-process communication? (If not then I cannot use it
for my image processing/detection part. If yes then it's a candidate)

Tim: So are sockets truly cross-platform and expose events and
variables between applications regardless of platform? How does it
compare to CORBA and why wouldn't I use that? Or is it the same thing?
I particularly want to easily reach 3rd party web apps via browser-add
ons (that I will write e.g. I know Mozilla uses XPCOM, and IE uses
ActiveX). So what's the most convenient single IPC stream that I
should have running?

Regardless of the architecture I choose, am I right that I would
definitely need an .exe running to kick-start and run the DirectShow
filter? There's no way of a filter "self-running" all the time, right?
So I have another stupid beginner question: can I have this
application instantiate automatically when the device is plugged in,
and which stops when they unplug it? (Additionally, can I make it
restart every time they force-kill the process in "Task Manager"?)

On Aug 6, 7:28 am, "Maxim S. Shatskih"

Post by Maxim S. Shatskih

Post by Navigateur
How would I make the 2 specific cameras be recognized and registered
as one (specific) device?

You need to design a way to merge the 2 video streams first.
Then, this "merger" should be implemented as one more DirectShow filter.

Post by Navigateur
Could I still use UVC drivers + a custom KS
proxy DirectShow plug-in?

Yes.

Post by Navigateur
DirectShow events, CORBA

How is this related? isn't this a Linux replacement for SOAP and similar things? how is it related to video?
Surely DirectShow supports properties on filters.
--
Maxim S. Shatskih
Windows DDK MVP

[Maxim S. Shatskih]

Post by Navigateur
Maxim: by events/variables, I mean my own basic detection events/
variables fired by me, not video events. Can DS filters perform bi-

Write your app which will host a DS graph, and expose OA events and methods from it.

Post by Navigateur
Regardless of the architecture I choose, am I right that I would
definitely need an .exe running to kick-start and run the DirectShow
filter?

Correct.

Post by Navigateur
So I have another stupid beginner question: can I have this
application instantiate automatically when the device is plugged in,

Write an additional service/Run key app which will monitor the device arrival events, or start the app from the coinstaller DLL.

Post by Navigateur
and which stops when they unplug it? (Additionally, can I make it
restart every time they force-kill the process in "Task Manager"?)

This would be a major misfeature. If the user wants to kill the app, he must be able to do this.

[Navigateur]

What are OA events?

Post by Maxim S. Shatskih
Write your app which will host a DS graph, and expose OA events and methods from it.

[Maxim S. Shatskih]

OLE Automation events

[Tim Roberts]

Post by Navigateur
What's your recommended overall architecture for this?
How would I make the 2 specific cameras be recognized and registered
as one (specific) device?

You wouldn't. You would have an APPLICATION connect to those two cameras
in standard DirectShow graphs, and have the application expose services to
clients. Perhaps as a COM server, or a network server, or some other
interprocess communication stream.

Post by Navigateur
Could I still use UVC drivers + a custom KS proxy DirectShow plug-in?

Well, if you don't want to provide streaming data from these cameras, why
would you consider combining the two in KS?

Really, that's a bad idea to begin with.

Post by Navigateur
For events/variables, should I use
DirectShow events, CORBA or something else? (given that I want the
events/variables to be easily available to multiple applications
running on all platforms, not only by native Windows apps).

To go cross-platform, you're pretty much locked in to sockets.

[Navigateur]

So what's the most convenient single IPC stream that I should have
running for all third-party app developers to use (including web
apps)? This seems the trickiest and most important choice that I have
to make.

How would I use sockets in the most convenient and cross-platform way?
Is it better than using OLE Automation? CORBA? XPCOM? Can I use all at
the same time or would it slow the application down?

Post by Navigateur
What's your recommended overall architecture for this?
How would I make the 2 specific cameras be recognized and registered
as one (specific) device?

You wouldn't.  You would have an APPLICATION connect to those two cameras
in standard DirectShow graphs, and have the application expose services to
clients.  Perhaps as a COM server, or a network server, or some other
interprocess communication stream.

Post by Navigateur
Could I still use UVC drivers + a custom KS proxy DirectShow plug-in?

Well, if you don't want to provide streaming data from these cameras, why
would you consider combining the two in KS?
Really, that's a bad idea to begin with.

Post by Navigateur
For events/variables, should I use
DirectShow events, CORBA or something else? (given that I want the
events/variables to be easily available to multiple applications
running on all platforms, not only by native Windows apps).

To go cross-platform, you're pretty much locked in to sockets.
--
Providenza & Boekelheide, Inc.

[Maxim S. Shatskih]

Post by Navigateur
How would I use sockets in the most convenient and cross-platform way?

Sockets are too low a level.

I would prefer HTTP+XML+SOAP. It is IIRC not-so-hard to be made compatible with DCOM and CORBA.

[Navigateur]

Very interesting! Looking into it it does seem programatically
verbose. Would it be more so than, say XPCOM and OLE Automation?

None of the solutions seem 100% suitable for what I'm doing. Would it
be worth making my own compact custom solution with my own data types?
What would that require me to make in total?

On Aug 7, 3:10 pm, "Maxim S. Shatskih"

Post by Maxim S. Shatskih

Post by Navigateur
How would I use sockets in the most convenient and cross-platform way?

Sockets are too low a level.
I would prefer HTTP+XML+SOAP. It is IIRC not-so-hard to be made compatible with DCOM and CORBA.
--
Maxim S. Shatskih
Windows DDK MVP

[Maxim S. Shatskih]

Post by Navigateur
be worth making my own compact custom solution with my own data types?

Depends on your platform.

With OA, you're tied to Windows. There is no OA in UNIXen.

With CORBA, you're tied _to particular CORBA package_ (under any popular UNIX and maybe even Windows) - the standard is too vague and different implementations are incompatible in terms of APIs (compatibe in terms of network protocol though).

[Navigateur]

I meant making my own simple (bi-directional) method invocation
"standard" for my app, using TCP? Tim said sockets are cross-platform,
right? What, in total, is the minimum I need to make for this to work?

On Aug 8, 1:33 pm, "Maxim S. Shatskih"

Post by Maxim S. Shatskih

Post by Navigateur
be worth making my own compact custom solution with my own data types?

Depends on your platform.
With OA, you're tied to Windows. There is no OA in UNIXen.
With CORBA, you're tied _to particular CORBA package_ (under any popular UNIX and maybe even Windows) - the standard is too vague and different implementations are incompatible in terms of APIs (compatibe in terms of network protocol though).
--
Maxim S. Shatskih
Windows DDK MVP

[m]

This is a situation where it will be mush easier in the long run if you use
a standard package / middleware instead of rolling your own. In the short
term, the features that the industry standard offerings have may seem
overkill, but if you roll you own, you will have to implement them sooner or
later. And the work to translate network to host byte order pales in
comparison with the work required to map code pages on non-ASCII systems; as
I have the misfortune of experiencing.

I echo Maxim's suggestion of XML / SOAP as the simplest, but COBERA & MQ are
also viable. In all cases though, you should not attempt this in KM, but
pass this work off to a UM service. All of these products use sockets
underneath, since socketys constitute the basis of all modern networking,
but abstract the binary details from you and allow you to focus on the data
that you want sent.

Post by Navigateur
I meant making my own simple (bi-directional) method invocation
"standard" for my app, using TCP? Tim said sockets are cross-platform,
right? What, in total, is the minimum I need to make for this to work?
On Aug 8, 1:33 pm, "Maxim S. Shatskih"

Post by Maxim S. Shatskih

Post by Navigateur
be worth making my own compact custom solution with my own data types?

Depends on your platform.
With OA, you're tied to Windows. There is no OA in UNIXen.
With CORBA, you're tied _to particular CORBA package_ (under any popular
UNIX and maybe even Windows) - the standard is too vague and different
implementations are incompatible in terms of APIs (compatibe in terms of
network protocol though).
--
Maxim S. Shatskih
Windows DDK MVP