synchronous emscripten_wget_data() with asyncify?

[Floh]

Hi,

for some reason I didn't pay that much attention to the asyncify and
emscripten_yield() stuff since in the small, new experimental engine code I
wrote more recently I avoided the need for blocking by design. But I'd like
to go back attempting to port big legacy code bases, and synchronous IO
calls was one of the main-'blockers' (excuse the pun).

What I would actually need is some synchronous version of
emscripten_wget_data() which downloads raw bytes without putting them into
the 'virtual file system'. Is this planned for the near future, or should I
attempt to write my own? If the latter, any tips and experiences to share?

I'm not sure what the API would look like, I think the function can still
have the onload and onerror callbacks, but would only return after either
of the callback had been called?

Cheers,
-Floh.

[Alon Zakai]

One option here is to do a synchronous XHR. This will not work on binary
data though, so it adds overhead, and there are cross-browser compatibility
issues.

The other is to use something like asyncify (experimental but works now) or
the emterpreter (experimental and doesn't work yet). Both add significant
overhead though, so the first option might be better.

- Alon

Post by Floh
Hi,
for some reason I didn't pay that much attention to the asyncify and
emscripten_yield() stuff since in the small, new experimental engine code I
wrote more recently I avoided the need for blocking by design. But I'd like
to go back attempting to port big legacy code bases, and synchronous IO
calls was one of the main-'blockers' (excuse the pun).
What I would actually need is some synchronous version of
emscripten_wget_data() which downloads raw bytes without putting them into
the 'virtual file system'. Is this planned for the near future, or should I
attempt to write my own? If the latter, any tips and experiences to share?
I'm not sure what the API would look like, I think the function can still
have the onload and onerror callbacks, but would only return after either
of the callback had been called?
Cheers,
-Floh.
--
You received this message because you are subscribed to the Google Groups
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[Floh]

Won't a synchronous XHR block/freeze the browser loop though? This is for
potential multi-second downloads. As far as I understood asyncify it allows
to return control to the browser loop and resumes at the specific position
down in the call-stack?

Will this add general performance overhead for all code, or only when and
where an 'asyncify block' is active?

Thanks,
-Floh.

Post by Alon Zakai
One option here is to do a synchronous XHR. This will not work on binary
data though, so it adds overhead, and there are cross-browser compatibility
issues.
The other is to use something like asyncify (experimental but works now)
or the emterpreter (experimental and doesn't work yet). Both add
significant overhead though, so the first option might be better.
- Alon

Post by Floh
Hi,
for some reason I didn't pay that much attention to the asyncify and
emscripten_yield() stuff since in the small, new experimental engine code I
wrote more recently I avoided the need for blocking by design. But I'd like
to go back attempting to port big legacy code bases, and synchronous IO
calls was one of the main-'blockers' (excuse the pun).
What I would actually need is some synchronous version of
emscripten_wget_data() which downloads raw bytes without putting them into
the 'virtual file system'. Is this planned for the near future, or should I
attempt to write my own? If the latter, any tips and experiences to share?
I'm not sure what the API would look like, I think the function can still
have the onload and onerror callbacks, but would only return after either
of the callback had been called?
Cheers,
-Floh.
--
You received this message because you are subscribed to the Google Groups
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[Alon Zakai]

Sync XHRs are complex, and work differently in different browsers. Some
might still show e.g. animations, but some might not, and all should not
allow other events to be handled. This is technically a deprecated feature
for these reasons.

Asyncify and emterpreter do have the ability to return control to the main
event loop, so animations and user events can continue normally (the app
needs to prevent event handlers from calling into compiled code while this
happens).

Asyncify increases code size substantially on methods that have an async
call, or may call such a function, and adds some speed slowdown. It also
has some known issues with things like setjmp and c++ exceptions, and in
general is experimental. Emterpreter on the other hand does not increase
code size (it decreases it actually), but it does run substantially slower.
The emterpreter currently works - i.e. we can compile into a bytecode, and
run that bytecode - but the specific feature to emulate sync calls is not
implemented yet. Should not be hard though, I just haven't had time; help
is welcome if you or anyone else likes hacking on that kind of stuff.

- Alon

Post by Floh
Won't a synchronous XHR block/freeze the browser loop though? This is for
potential multi-second downloads. As far as I understood asyncify it allows
to return control to the browser loop and resumes at the specific position
down in the call-stack?
Will this add general performance overhead for all code, or only when and
where an 'asyncify block' is active?
Thanks,
-Floh.

Post by Alon Zakai
One option here is to do a synchronous XHR. This will not work on binary
data though, so it adds overhead, and there are cross-browser compatibility
issues.
The other is to use something like asyncify (experimental but works now)
or the emterpreter (experimental and doesn't work yet). Both add
significant overhead though, so the first option might be better.
- Alon

Post by Floh
Hi,
for some reason I didn't pay that much attention to the asyncify and
emscripten_yield() stuff since in the small, new experimental engine code I
wrote more recently I avoided the need for blocking by design. But I'd like
to go back attempting to port big legacy code bases, and synchronous IO
calls was one of the main-'blockers' (excuse the pun).
What I would actually need is some synchronous version of
emscripten_wget_data() which downloads raw bytes without putting them into
the 'virtual file system'. Is this planned for the near future, or should I
attempt to write my own? If the latter, any tips and experiences to share?
I'm not sure what the API would look like, I think the function can
still have the onload and onerror callbacks, but would only return after
either of the callback had been called?
Cheers,
-Floh.
--
You received this message because you are subscribed to the Google
Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[Floh]

Thanks for the explanation! I think I'll first play around with asyncify a
bit to get a feeling for the size and speed overhead.

Do you know how big the emterpreter size overhead will be for the actual
interpreter implementation (I guess that's statically linked in?). Dozens
or rather hundreds of kByte?

Cheers,
-Floh.

Post by Alon Zakai
Sync XHRs are complex, and work differently in different browsers. Some
might still show e.g. animations, but some might not, and all should not
allow other events to be handled. This is technically a deprecated feature
for these reasons.
Asyncify and emterpreter do have the ability to return control to the main
event loop, so animations and user events can continue normally (the app
needs to prevent event handlers from calling into compiled code while this
happens).
Asyncify increases code size substantially on methods that have an async
call, or may call such a function, and adds some speed slowdown. It also
has some known issues with things like setjmp and c++ exceptions, and in
general is experimental. Emterpreter on the other hand does not increase
code size (it decreases it actually), but it does run substantially slower.
The emterpreter currently works - i.e. we can compile into a bytecode, and
run that bytecode - but the specific feature to emulate sync calls is not
implemented yet. Should not be hard though, I just haven't had time; help
is welcome if you or anyone else likes hacking on that kind of stuff.
- Alon

Post by Floh
Won't a synchronous XHR block/freeze the browser loop though? This is for
potential multi-second downloads. As far as I understood asyncify it allows
to return control to the browser loop and resumes at the specific position
down in the call-stack?
Will this add general performance overhead for all code, or only when and
where an 'asyncify block' is active?
Thanks,
-Floh.

Post by Alon Zakai
One option here is to do a synchronous XHR. This will not work on binary
data though, so it adds overhead, and there are cross-browser compatibility
issues.
The other is to use something like asyncify (experimental but works now)
or the emterpreter (experimental and doesn't work yet). Both add
significant overhead though, so the first option might be better.
- Alon

Post by Floh
Hi,
for some reason I didn't pay that much attention to the asyncify and
emscripten_yield() stuff since in the small, new experimental engine code I
wrote more recently I avoided the need for blocking by design. But I'd like
to go back attempting to port big legacy code bases, and synchronous IO
calls was one of the main-'blockers' (excuse the pun).
What I would actually need is some synchronous version of
emscripten_wget_data() which downloads raw bytes without putting them into
the 'virtual file system'. Is this planned for the near future, or should I
attempt to write my own? If the latter, any tips and experiences to share?
I'm not sure what the API would look like, I think the function can
still have the onload and onerror callbacks, but would only return after
either of the callback had been called?
Cheers,
-Floh.
--
You received this message because you are subscribed to the Google
Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[Alon Zakai]

Interpreter is statically linked in, yes. It's small in general, about 100
opcodes each taking 1 line or so. It also needs a line for each
non-interpreted function being called though. And the binary code replacing
the interpreted code is much smaller than source (although similar after
gziping them both).

- Alon

Post by Floh
Thanks for the explanation! I think I'll first play around with asyncify a
bit to get a feeling for the size and speed overhead.
Do you know how big the emterpreter size overhead will be for the actual
interpreter implementation (I guess that's statically linked in?). Dozens
or rather hundreds of kByte?
Cheers,
-Floh.

Post by Alon Zakai
Sync XHRs are complex, and work differently in different browsers. Some
might still show e.g. animations, but some might not, and all should not
allow other events to be handled. This is technically a deprecated feature
for these reasons.
Asyncify and emterpreter do have the ability to return control to the
main event loop, so animations and user events can continue normally (the
app needs to prevent event handlers from calling into compiled code while
this happens).
Asyncify increases code size substantially on methods that have an async
call, or may call such a function, and adds some speed slowdown. It also
has some known issues with things like setjmp and c++ exceptions, and in
general is experimental. Emterpreter on the other hand does not increase
code size (it decreases it actually), but it does run substantially slower.
The emterpreter currently works - i.e. we can compile into a bytecode, and
run that bytecode - but the specific feature to emulate sync calls is not
implemented yet. Should not be hard though, I just haven't had time; help
is welcome if you or anyone else likes hacking on that kind of stuff.
- Alon

Post by Floh
Won't a synchronous XHR block/freeze the browser loop though? This is
for potential multi-second downloads. As far as I understood asyncify it
allows to return control to the browser loop and resumes at the specific
position down in the call-stack?
Will this add general performance overhead for all code, or only when
and where an 'asyncify block' is active?
Thanks,
-Floh.

Post by Alon Zakai
One option here is to do a synchronous XHR. This will not work on
binary data though, so it adds overhead, and there are cross-browser
compatibility issues.
The other is to use something like asyncify (experimental but works
now) or the emterpreter (experimental and doesn't work yet). Both add
significant overhead though, so the first option might be better.
- Alon

Post by Floh
Hi,
for some reason I didn't pay that much attention to the asyncify and
emscripten_yield() stuff since in the small, new experimental engine code I
wrote more recently I avoided the need for blocking by design. But I'd like
to go back attempting to port big legacy code bases, and synchronous IO
calls was one of the main-'blockers' (excuse the pun).
What I would actually need is some synchronous version of
emscripten_wget_data() which downloads raw bytes without putting them into
the 'virtual file system'. Is this planned for the near future, or should I
attempt to write my own? If the latter, any tips and experiences to share?
I'm not sure what the API would look like, I think the function can
still have the onload and onerror callbacks, but would only return after
either of the callback had been called?
Cheers,
-Floh.
--
You received this message because you are subscribed to the Google
Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google
Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[Floh]

Sounds good, looking forward to playing around with it :)

Post by Alon Zakai
Interpreter is statically linked in, yes. It's small in general, about 100
opcodes each taking 1 line or so. It also needs a line for each
non-interpreted function being called though. And the binary code replacing
the interpreted code is much smaller than source (although similar after
gziping them both).
- Alon

Post by Floh
Thanks for the explanation! I think I'll first play around with asyncify
a bit to get a feeling for the size and speed overhead.
Do you know how big the emterpreter size overhead will be for the actual
interpreter implementation (I guess that's statically linked in?). Dozens
or rather hundreds of kByte?
Cheers,
-Floh.

Post by Alon Zakai
Sync XHRs are complex, and work differently in different browsers. Some
might still show e.g. animations, but some might not, and all should not
allow other events to be handled. This is technically a deprecated feature
for these reasons.
Asyncify and emterpreter do have the ability to return control to the
main event loop, so animations and user events can continue normally (the
app needs to prevent event handlers from calling into compiled code while
this happens).
Asyncify increases code size substantially on methods that have an async
call, or may call such a function, and adds some speed slowdown. It also
has some known issues with things like setjmp and c++ exceptions, and in
general is experimental. Emterpreter on the other hand does not increase
code size (it decreases it actually), but it does run substantially slower.
The emterpreter currently works - i.e. we can compile into a bytecode, and
run that bytecode - but the specific feature to emulate sync calls is not
implemented yet. Should not be hard though, I just haven't had time; help
is welcome if you or anyone else likes hacking on that kind of stuff.
- Alon

Post by Floh
Won't a synchronous XHR block/freeze the browser loop though? This is
for potential multi-second downloads. As far as I understood asyncify it
allows to return control to the browser loop and resumes at the specific
position down in the call-stack?
Will this add general performance overhead for all code, or only when
and where an 'asyncify block' is active?
Thanks,
-Floh.

Post by Alon Zakai
One option here is to do a synchronous XHR. This will not work on
binary data though, so it adds overhead, and there are cross-browser
compatibility issues.
The other is to use something like asyncify (experimental but works
now) or the emterpreter (experimental and doesn't work yet). Both add
significant overhead though, so the first option might be better.
- Alon

Post by Floh
Hi,
for some reason I didn't pay that much attention to the asyncify and
emscripten_yield() stuff since in the small, new experimental engine code I
wrote more recently I avoided the need for blocking by design. But I'd like
to go back attempting to port big legacy code bases, and synchronous IO
calls was one of the main-'blockers' (excuse the pun).
What I would actually need is some synchronous version of
emscripten_wget_data() which downloads raw bytes without putting them into
the 'virtual file system'. Is this planned for the near future, or should I
attempt to write my own? If the latter, any tips and experiences to share?
I'm not sure what the API would look like, I think the function can
still have the onload and onerror callbacks, but would only return after
either of the callback had been called?
Cheers,
-Floh.
--
You received this message because you are subscribed to the Google
Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it,
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google
Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.