Feed Aggregator Page 643
Rendered on Sat, 05 Sep 2020 20:33:18 GMT
Rendered on Sat, 05 Sep 2020 20:33:18 GMT
via Planet Lisp by on Sat, 05 Sep 2020 18:07:16 GMT
It turns out that sound processing is pretty difficult. I've been hacking away at an almost from-scratch rewrite of Shirakumo's sound systems for the past few months and it's caused a lot of anguish.
Now, Harmony is not a new system. It dates back quite a few years, and I had even written another article on it previously. However, this previous version had several serious flaws and problems, some of which penetrating all the way through the audio stack. With the rewrite I'm however now much more confident in the quality and usability of the system.
First though, a bit of terminology: in digital audio processing, audio is represented as a sequence of samples; regularly recorded amplitudes of the audio signal. These samples are recorded at a constant rate, the "sample rate," which often is either 44.1kHz or 48kHz. Often each sample is represented as a float going from -1 to +1, and multiple such sample sequences are combined to form the signal for as many channels as you need (stereo, surround, etc.) When processing audio data, a limited sequence of samples is kept in a buffer, which processors can then operate on.
The initial problem with the system was one of resampling: the system was written with the assumption that one could keep a constant samplerate throughout the entire audio processing pipeline. This, however, turned out to not be suitable. The issue manifested itself on Windows, where the output backend could require a different samplerate to the one the system was initially configured for. Thus, at least at the end-points, resampling would be required.
This immediately lead to another problem though: the system was also written with the assumption that every part of the pipeline could consume and produce a full audio buffer every time it was run. However, with resampling, border issues appear and it's not always possible to consume the full input buffer. This issue permeates throughout the processing pipeline, as now the final processor cannot consume all data, and so when the system is run next, the processor before the last cannot produce a full buffer as it would overwrite data.
Ultimately though, the fixed samplerate and fixed buffer size design lead to a restriction that made it impossible to represent certain effects like a speed change, which would produce and consume samples at different rates. And so, pretty much everything had to be rewritten to work with this in mind. To spare you the troublesome process of figuring out a design, let's just jump to what the system is like now:
At the most basic level resides the bip-buffer
interface, which implements a lockless bipartite buffer. It's lockless so that one thread can write, and another can read from it simultaneously. It's bipartite so that the regions it hands out are always consecutive regions of memory, rather than wrapping around like in a ring buffer. This interface is implemented by buffer
s and pack
s. buffer
s represent internal audio samples of one channel in float
format, whereas pack
s represent external audio samples in any format, with any number of channels.
Then there's the parts that actually perform audio processing. These are called segment
s, and follow a generic interface that allows them to do their work, and also allows them to be introspected. Namely they each have a number of input fields, a number of output fields, and a number of parameter fields. To the input and output fields you can attach a buffer
, which will cause the segment
s to exchange data. Assembling a network is then just a matter of creating the segment
s, creating a buffer
for each connection, and then setting them at the appropriate in/out fields.
At the endpoints, where you need to exchange data with other systems such as file decoders or device drivers, you'll probably want to make use of the unpacker
and packer
segments, which perform the necessary encoding to translate between the float buffer
s and the compact pack
s. These segments will also perform sample rate conversion as necessary.
Since we have proper bip buffers connecting everything, a segment
can now consume and produce at a variable rate without needing to be aware of the rates going on in the rest of the system. The rates will automatically propagate through the system as the buffers are updated.
Now, all of this behaviour, including many practical standard segment
s are implemented in a C library called libmixed. Audio has some pretty severe latency restrictions, and that's why, with great pain, I decided to implement the bulk of the audio processing in C, rather than Lisp. This has cost me a lot of time, but I still think the performance gains are worth it, or I would have had to spend similar, if not more time, trying to match the performance with Lisp code. I hope that this kind of thing will no longer be necessary at some point in the future, but for now this is where we are.
Anyway, being implemented in C also means it can be useful for people outside of Lisp, and I really do hope that others will take advantage of libmixed, as I think it has a lot of useful work behind it. To my knowledge there's currently no free (as in BSD) and capable audio processing system out there. The library also offers a plugin and reflection/introspection API so that one could build a GUI that can represent segments and buffers in a very generic fashion, allowing users to easily plug together processing networks.
Now, one level above libmixed sits cl-mixed, the Lisp bindings library that takes care of the low level stuff and wraps it all in a nice Lisp interface. It also takes care of offering some support structures where needed, such as managing the input locations when dealing with variable input segments such as mixers. It also offers a ton of extension systems for interacting with various file formats and playback backends:
ALSA
Linux playbackCoreAudio
macOS playbackFLAC
FLAC file decodingJack
JackAudio playbackOSS
OSS playback (BSD)PulseAudio
Linux desktop playbackSDL2
SDL2 integration if you're already using SDL2WASAPI
Windows Vista+ playbackWAV
WAV file decodingWinMM
Windows 3.0+ playbackXAudio2
Windows 8+ playbackmpg123
MP3 decodingout123
Cross-platform playback (C blob)I'd like to add more decoders, and at some point also input for the various operating system backends, but for now this is more than plenty. Some of the backends still have issues (WinMM, XAudio2, CoreAudio), which I have spent a long time trying to figure out already, so far unsuccessful. I'm not too bothered about WinMM and XAudio2, but CoreAudio definitely needs to be made to work properly soon.
The reason these backends are implemented in Lisp is so that there's no additional dependencies on shared libraries that might be versioned and interact poorly when deployed. Since the actual work performed in their respective segment amounts to requesting a buffer region and performing one call, the performance impact from it should also be entirely negligible.
cl-mixed also offers a virtual
segment that allows you to implement a segment in Lisp and integrate it into a standard pipeline. This is possible thanks to the standardised architecture in libmixed, and can be very useful to experiment with effects very quickly. If I ever intend on developing a new effects segment, I'll definitely implement it in Lisp first to take advantage of rapid prototyping, before lowering it down to C if performance should become an issue.
On that note, cl-mixed actually uses static-vectors to implement the backing storage of pack
s and buffer
s, as well as all of the bip-buffer protocol. This means that you can interact with packs and buffers from Lisp as if they were normal Lisp arrays, without ever having to worry about FFI.
That said, cl-mixed will not do buffer management or resource management in general for you. You'll still have to manually create and free segments and buffers and make sure they're connected. You'll also have to run the mixing loop yourself and make sure you do that often enough to not cause stuttering.
This is where Harmony steps in. Being the high-level component, it imposes a bit of architecture on you, but in turn takes care of a lot of lower level plumbing. In effect, with Harmony you can perform playback as easily as:
(harmony:start (harmony:make-simple-server))
(harmony:play "music.mp3" :mixer :music :loop T)
(harmony:play "effect.wav" :mixer :effect :location '(10 0 0))
It'll take care of detecting the appropriate backend for your platform, setting up channel conversion and basic mixing infrastructure, allocating and re-using buffers, automatically cleaning up when a sound source ends, and performing low-latency audio processing in the background.
It can also do fun stuff like automatically creating a network to apply effects to a source.
(harmony:play "music.wav" :mixer :music :effects
'((mixed:speed-change :speed-factor 2.0)
(mixed:pitch :pitch 0.5)))
Which would play the music at double the speed, but with a pitch correction applied so that the notes should still be the correct frequency.
Hopefully this will make it easy enough to use for games without having to worry about all the low level detail aspects. I'm going to find out how well this all works soon, as it's now at a stable enough state that I can start working it into Kandria.
If you're interested in using these systems or contributing to them, let me know! I'd be happy to provide assistance.
If you like my work in general and want to donate, you can do that too, either on GitHub Sponsors for recurring donations, or on Ko-Fi for one-time donations.
Thanks for reading!
via Elm - Latest posts by @system system on Sat, 05 Sep 2020 18:57:29 GMT
This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.
via Elm - Latest posts by @surferjeff Jeffrey Rennie on Sat, 05 Sep 2020 14:43:50 GMT
Thank you Hans-Helmut! That worked!
via Elm - Latest posts by @MartinS on Sat, 05 Sep 2020 14:09:29 GMT
If you want to store data in a url and the format doesn’t matter (i.e. you’re okay with domain.com/<nonsense text>
) then elm-serialize could work. In this case it looks like the url needs to have user readable paths so this wouldn’t work.
Edit: Codecs (generating both the encoder and decoder from a single function) could be an useful approach to handling urls though. I haven’t heard of any attempts to do this. Not sure if that’s because no one has bothered or if people have tried and it didn’t work.
via Elm - Latest posts by @sebbes Sébastien Besnier on Sat, 05 Sep 2020 13:58:47 GMT
Maybe https://package.elm-lang.org/packages/MartinSStewart/elm-serialize/1.2.1/ can help, but I’ve never used it. @MartinS could maybe give more details.
via Elm - Latest posts by @sebbes Sébastien Besnier on Sat, 05 Sep 2020 13:53:49 GMT
I also encountered this situation (almost everyone writing this kind of Route module does!).
What I do for now is to write (by hand) a routeFuzzer : Fuzzer Route
being in charge of randomly generate all possible routes and then I only have one test being:
fuzz routeFuzzer
"Test route serialization/parsing matching"
(\route ->
("http://domain.com" ++ Route.toString route)
|> Url.fromString
|> Maybe.andThen Route.parse
|> Expect.equal (Just route)
)
This is a bit painful to write but if you change something in your route structure, this won’t compile and you cannot miss that. The only situation where you have to be very careful is when you add a new route (but this doesn’t happen very often).
via Elm - Latest posts by @sebbes Sébastien Besnier on Sat, 05 Sep 2020 13:47:16 GMT
Ok, my bad! Thank you @luke for correcting me.
I don’t know why I thought those functions was used during port process. I’ve checked what I’ve done and I don’t use these functions neither.
So my claim was wrong.
In the case we would want to use typescript to get rid of the encoding part, I wonder if we should care about “copy” the function argument in order to prevent mutation by the caller. If we do:
arg = { b: 5};
res = elmModule.f(arg);
arg.b = 4;
this should not bother f
. But is it possible for arg
to be modified in another place (say it is a global variable), during the execution of f
?
via Elm - Latest posts by @eimfach Robin G. on Sat, 05 Sep 2020 11:18:01 GMT
Is this documented somewhere? Looks like a missing piece for Elms (Ports) documentation
via Elm - Latest posts by @system system on Sat, 05 Sep 2020 11:11:37 GMT
This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.
via Elm - Latest posts by @hans-helmut Hans-Helmut on Sat, 05 Sep 2020 10:41:10 GMT
By conditional adding empty lists:
Canvas.shapes
[ -- Settings
Canvas.Settings.Line.lineWidth 5
, Canvas.Settings.Line.lineCap Canvas.Settings.Line.RoundCap
, Canvas.Settings.stroke Color.black
]
( [ -- Shapes
path ( 2, 297 )
]
++
if (model.turns > 10 )then
-- Draw the stand.
[ lineTo ( 297, 297 )
, moveTo ( 13, 297 )
, lineTo ( 13, 3 )
, lineTo ( 150, 3 )
, lineTo ( 150, 25 )
]
else
[]
++ ...
via Elm - Latest posts by @surferjeff Jeffrey Rennie on Sat, 05 Sep 2020 04:11:29 GMT
I’m building a simple hangman game to learn elm, translating it from some old javascript code.
To draw the hangman with a canvas, I found https://package.elm-lang.org/packages/joakin/elm-canvas/latest/ which is working out pretty well.
However, I need to conditionally draw parts of the body depending on how many turns the player has left.
Here’s my old Javacript code:
let ctx = canvas.getContext('2d');
ctx.beginPath();
ctx.fillStyle = "white";
ctx.fillRect(0, 0, 300, 300);
ctx.strokeStyle = "black";
ctx.lineWidth = 5;
ctx.lineCap = "round";
// Draw the stand.
ctx.moveTo(3, 297);
ctx.lineTo(297, 297);
ctx.moveTo(13, 297);
ctx.lineTo(13, 3);
ctx.lineTo(150, 3);
ctx.lineTo(150, 25);
let turns:number = game.getTurnsLeft();
if (turns < 8) {
// Draw the head.
ctx.moveTo(150, 25);
ctx.arc(150, 50, 25, -Math.PI / 2, 2 * Math.PI);
}
if (turns < 7) {
// Draw the body.
ctx.moveTo(150, 75);
ctx.lineTo(150, 180);
}
if (turns < 6) {
// Draw left arm.
ctx.moveTo(150, 90);
ctx.lineTo(90, 150);
}
if (turns < 5) {
// Draw right arm.
ctx.moveTo(150, 90);
ctx.lineTo(210, 150);
}
if (turns < 4) {
// Draw left leg.
ctx.moveTo(150, 180);
ctx.lineTo(90, 270);
}
if (turns < 3) {
// Draw right leg.
ctx.moveTo(150, 180);
ctx.lineTo(210, 270);
}
if (turns < 2) {
// Draw left foot.
ctx.moveTo(90, 270);
ctx.lineTo(70, 260);
}
if (turns < 1) {
// Draw right foot.
ctx.moveTo(210, 270);
ctx.lineTo(230, 260);
}
ctx.stroke();
And here’s my elm code. How do I add something like an if {}
statement to only draw parts of the man depending on how many turns the player has left?
Canvas.shapes
[ -- Settings
Canvas.Settings.Line.lineWidth 5
, Canvas.Settings.Line.lineCap Canvas.Settings.Line.RoundCap
, Canvas.Settings.stroke Color.black
]
[ -- Shapes
path ( 2, 297 )
-- Draw the stand.
[ lineTo ( 297, 297 )
, moveTo ( 13, 297 )
, lineTo ( 13, 3 )
, lineTo ( 150, 3 )
, lineTo ( 150, 25 )
]
-- Draw the head
, circle ( 150, 50 ) 25
, path ( 150, 75 )
-- Draw the body
[ lineTo ( 150, 180 )
-- Draw left arm
, moveTo ( 150, 90 )
, lineTo ( 90, 150 )
-- Draw right arm
, moveTo ( 150, 90 )
, lineTo ( 210, 150 )
-- Draw left leg.
, moveTo ( 150, 180 )
, lineTo ( 90, 270 )
-- Draw right leg.
, moveTo ( 150, 180 )
, lineTo ( 210, 270 )
-- Draw left foot.
, moveTo ( 90, 270 )
, lineTo ( 70, 260 )
-- Draw right foot.
, moveTo ( 210, 270 )
, lineTo ( 230, 260 )
]
]
via Elm - Latest posts by @rkb Robert K Bell on Fri, 04 Sep 2020 23:58:53 GMT
The problem: Given a big, hairy, often-changing Route
custom type, and a pair of functions Route -> String
and String -> Maybe Route
, how can we be sure that (almost) every Route turned into a string and back into Route produces the same value that we started with?
Route
looks like:
module Route exposing (..)
import UserRoutes exposing UserRoutes
import TaskRoutes exposing TaskRoutes
type Route
= Home
| Settings
| About
| User String UserRoutes
| Task Int TaskRoutes
-- ...dozens more...
Is there some tool I can point at Route.elm, say “give me the structure of the Route
type”, and have it follow the module imports, so that it returns an enumerable list of properties? Something like:
{
type: "CustomType",
name: "Route",
members: [
{
name: "Home",
},
{
name: "Settings",
},
{
name: "About",
},
{
name: "User",
args: [
{
type: "String",
},
{
type: "CustomType",
name: "UserRoutes",
members: [
{
name: "Summary",
},
{
name: "Activity",
},
// ... etc ...
],
}
],
},
{
name: "Task",
args: [
{
type: "Int",
},
{
type: "CustomType",
name: "TaskRoutes",
members: [
{
name: "Active",
},
{
name: "Overdue",
},
// ... etc ...
],
}
],
},
// ... etc ...
]
}
I’m thinking that I’d use that to generate a RouteTest.elm file that uses default (or fuzzed) values for the Ints and Strings, which might look like:
suite : Test
suite =
describe "routes"
[ test "should parse generated cases" <|
\_ ->
let
data =
[ Home
, Settings
, About
, User "a" UserRoutes.Summary
, User "a" UserRoutes.Activity
, Task 1 TaskRoutes.Active
, Task 1 TaskRoutes.Overdue
-- ... plus the rest ...
]
actual =
data |> List.map (routeToString >> routeFromString)
expected =
data |> List.map Just
in
actual |> Expect.equalLists expected
]
via Planet Lisp by on Fri, 04 Sep 2020 17:52:13 GMT
This system is pretty old. It does not have its own revision control and is hosted here. It provides the replacement for the defpackage
macro. This replacement makes easy to "inherit" your package from another and to replace some symbols with your own.
For example, @stylewarning's cl-generic-arithmetic
uses it to redefine some functions from cl
package. It defines a new package which uses cl
and reexports all symbols except some which are defined in by the cl-generic-arithmetic
as generic functions.
Let's repeat this to make +
a function generic!
POFTHEDAY> (org.tfeb.clc:defpackage cl-generic
(:extends/excluding #:cl
#:+)
(:export #:+))
;; For simplicity, I'll define this operation as
;; binary. But for real implementation it should
;; support variadic arguments.
POFTHEDAY> (defgeneric cl-generic:+ (left right)
(:method ((left number) (right number))
(cl:+ left right))
(:method ((left string) (right string))
(concatenate 'string left right))
(:method ((left string) (right number))
(format nil "~A~A" left right)))
Now we can define another package which will use this generic function. Note, I'll just :use
this new package instead of standard :cl
package:
POFTHEDAY> (defpackage foo
(:use :cl-generic))
POFTHEDAY> (in-package foo)
FOO> (+ 1 2)
3
FOO> (+ "Hello " "World!")
"Hello World!"
;; Other function are standard, becase they are inherited
;; from the standard package:
FOO> (- 5 3)
2
FOO> (- "Hello " "World!")
; Debugger entered on #<TYPE-ERROR expected-type: NUMBER datum: "Hello ">
By the way, you can get the same effect by using uiop:define-package
. But it will be a little bit wordy:
POFTHEDAY> (uiop:define-package cl-generic2
(:use #:cl)
(:shadow #:+)
(:reexport #:cl))
#<PACKAGE "CL-GENERIC2">
POFTHEDAY> (defgeneric cl-generic2:+ (left right)
(:method ((left number) (right number))
(cl:+ left right))
(:method ((left string) (right string))
(concatenate 'string left right))
(:method ((left string) (right number))
(format nil "~A~A" left right)))
POFTHEDAY> (defpackage foo2
(:use :cl-generic2))
POFTHEDAY> (in-package foo)
POFTHEDAY> (in-package foo2)
FOO2> (+ 1 2)
3
FOO2> (+ "Hello " "World!")
"Hello World!"
FOO2> (- 5 3)
2
FOO2> (- "Hello " "World!")
; Debugger entered on #<TYPE-ERROR expected-type: NUMBER datum: "Hello ">
Probably I missed some of the conduit-packages
features. Please, read its sources and tell me if you will find something interesting!
via Elm - Latest posts by @mradke Mradke on Fri, 04 Sep 2020 19:29:35 GMT
I cannot recommend libraries for Haskell backends from production experience, but I spent a bit time the last weeks figuring the same thing out for me.
For me the most interesting web framework seems to be servant since it has great docs and seems to be rather complete for API development. It is also a nice gimmick that it can automatically generate Elm decoder / encoder for your API.
For the database I tried out HDBC
and yeshql
, since I need access to a weird database through odbc
with a custom SQL dialect. I think I remember hearing good things about https://hackage.haskell.org/package/hasql when you’re using postgresql… Although these options are not ORMs.
Since I probably wouldn’t use GraphQL with servant
, I haven’t explored this, but https://morpheusgraphql.com/ looks really interesting, just in case you haven’t seen it yet.
via Elm - Latest posts by @luke Luke Westby on Fri, 04 Sep 2020 18:56:28 GMT
Neither of these lines are invoked when passing a value through a port. The first is called only when trying to parse a String
, like from the body of an HTTP response. There is also Json.Decode.decodeValue
which deals only with Value
s. The second line similarly is only invoked if you need to render a Value
to a String
by explicitly calling Json.Encode.encode
.
When the compiler generates code for ports it checks the type and sets up the appropriate converter from JS to Elm for that type. Lists and tuples, for example, get special treatment by getting converted to and from JS Arrays, but in the case of a Value
, the converter is identity
or Json.Decode.value
(the Decoder
analogue to identity
). No stringifying.
Generated port code for a Value
looks like this:
var $author$project$Module$portName = _Platform_outgoingPort('portName', $elm$core$Basics$identity);
var $author$project$Module$portName = _Platform_incomingPort('portName', $elm$json$Json$Decode$value);
Those generated arguments are used here https://github.com/elm/core/blob/master/src/Elm/Kernel/Platform.js#L363-L471
via Elm - Latest posts by @system system on Fri, 04 Sep 2020 16:50:02 GMT
This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.
via Elm - Latest posts by @system system on Fri, 04 Sep 2020 07:46:32 GMT
This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.
via Elm - Latest posts by @sebbes Sébastien Besnier on Fri, 04 Sep 2020 06:36:36 GMT
Well…
via Elm - Latest posts by @pd-andy Andy Thompson on Fri, 04 Sep 2020 05:58:01 GMT
Elm definitely doesn’t do this by default for ports. You can pass in any javascript value you like (stringifiable or not) and elm can handle it fine.
via Elm - Latest posts by @Laurent Laurent Payot on Fri, 04 Sep 2020 05:44:14 GMT
Doesn’t JSON.parse(JSON.stringify(...))
transform the object? https://medium.com/@pmzubar/why-json-parse-json-stringify-is-a-bad-practice-to-clone-an-object-in-javascript-b28ac5e36521