You will wear many hats depending on your interests & skillset. You'll be doing some combination of:
* Data engineering: figuring out how to efficiently get, clean up, cluster, and combine data.
* Building/fine-tuning LLMs to query data via natural language.
* Working on gSheets and Excel extensions.
* Working on front-end website and dashboards.
* Working on integrations with various enterprise tools.
* Interacting directly with customers (mostly executives at startups).
* Scaling our deployment, optimizing where and how we run.
Our tech stack:
* Python (pandas)
* Apache Airflow
* AWS
(To be clear, none of these things are pre-reqs. We want engineers who can quickly pick up new languages/technologies and want to keep our stack simple and mainstream enough to support that, while generally choosing the correct tool for the job.)
About us: Ldgr is building data pipelines to take messy data from a variety of sources ranging from modern APIs to crappy legacy systems. We clean it up and combine them all into one master data source with a pristine, easy-to-query schema. We sell this to businesses, who all have the same basic building blocks: customers, products, subscriptions, etc. The systems they use to track these things are completely disjoint and often impenetrable for non-technical users (e.g., the CEO). We’re building a universal system for organizing and making available all of a business’ information.
You will be one of the first employees, working directly with the founders (former YC founder and the founding engineer at MongoDB). We have also raised our first round of institutional financing and are well-capitalized for the next several years.
I'm hosting the machines so that users don't have to set anything up on the cloud side. It could have on prem or open source versions if people are interested in setting up their own clusters.
So far I'm implemented pretty coarse-grained control over machine specs: it's $20/month for a MacBook-like machine and $3/hour if you want to use "a GPU" (whatever Amazon has available for cheap). I figured the config could specify exact disk/ram/instance type requirements in the future, it sounds like something you're interested in?
Thanks! Yeah, it takes a look at your requirements.txt (or environment.yml) and uses that to install on the remote machine. So as long as that doesn't specify a particular arch, it'll chose whatever it's running on.
rx is a binary that you can use as a prefix to any other command (e.g., `rx ls` instead of `ls`) to run a command on the cloud. It sets up your project on the cloud and transparently copies what you're working on back and forth from your local machine.
Background: I was trying to build an app that used stable diffusion and got very frustrated by what should have been simple: I wanted to develop my program locally, but run it on a different machine. Setting up, copying over, and running the program ended up being an exercise in frustration (Careful documentation of the process: https://blog.run-rx.com/cloud-development-the-board-game).
To make life easier, I made a simple binary to copy everything to the cloud and back. Then I added streaming stdio and made it keep dependencies up-to-date and automatic installation of the languages the project uses and on and on.
Current state: give the CLI a project and it'll set up a persistent remote environment that is synced to your local environment (all dependencies installed and automatically kept up-to-date). This lets you develop locally using any tools you like, while still running remotely for anything where you don't want using your CPU.
Please give it a try, I'd love any and all feedback!
I would look at it as more of "allowing normal people access to unbefore-dreamed-of levels of draftsmanship" vs some comment on capital-A Art. It allows non-artists to express themselves visually.
I'm biased: I've been working on an image generation app. But the beta users I've had so far will generate fifty or a hundred images in a day. That isn't a use case traditional artists support.
I'm a developer on Bazel, and AFAIK there is no white paper. We definitely don't want to "throw it over the wall," we're going to try to push more and more development into the open over time.
Ah, OK, thanks. I do get a kick out of reading various Google white papers (GFS, Spanner, and the Multi-Paxos implementation for Chubby come to mind), so I'd definitely be interested in the prospect of a future white paper :).
The Golang source code for the server code at google is built with this tool. The rules that accomplish this are rather complex due to their interactions with our C++ libraries, and predates the open source "Go" tool. The experience with the Google internal rules, motivated some of the choices in the "go" tool, I believe.
It would be nice to make these semantics match the external ones better, but it requires us to open up more tooling, so people won't need to write BUILD files.
In what cases would using Bazel make sense to build Go projects? If they're extremely large? If they have a lot of dependencies on code in other languages? If you need sophisticated build/release tooling?
BTW, thanks for the release! Will have a fun time digging through this over the next few days. I heard some murmurs that Blaze was going to be open sourced from around the watercooler but didn't think it'd be so soon.
I guess if you want to integrate Go tools with builds in other languages. If you are using pure Go for your entire ecosystem, there is not much point in using Bazel, as the "go" tool is very capable for that scenario.
Check out http://bazel.io/docs/build-encyclopedia.html#maven_jar. In the root of your build, specify the jars you want from maven and then add them as dependencies in your BUILD files. The first time you run "bazel build", they'll be downloaded and cached from then on. It's somewhat limited in functionality at the moment, but should work for basic "download and depend on a jar".
* Suppose I develop using multiple repos and http_archive. I'd like to make changes both to a library and to a project that depends on it simultaneously, without committing the library patches to master github repo just yet. Is there a way to configure the http_archive, let's say by saying "bazel --mode=local", and have it customize the remote archive http to use a different url (say, my github's fork instead of the master github) for that build?
Scala support: yes, add using Skylark. Definitely let us know if you run into any rough edges, Skylark is a work-in-progress.
For multiple repos: there's no command line flag, but you could change the WORKSPACE file to use http://bazel.io/docs/build-encyclopedia.html#local_repositor.... Unfortunately, this may be of limited use to you. At the moment it's optimized/bugged to assumed that your local repos don't change, so it won't rebuild them (this is great for things like the JDK and gcc, but not so much for actual in-development repos). Feel free to file feature requests for any functionality you need, I'll be working on this a lot over the next couple months.
Company: theldgr.com
Location: NYC (onsite)
Contact: kristina@
You will wear many hats depending on your interests & skillset. You'll be doing some combination of:
Our tech stack: (To be clear, none of these things are pre-reqs. We want engineers who can quickly pick up new languages/technologies and want to keep our stack simple and mainstream enough to support that, while generally choosing the correct tool for the job.)About us: Ldgr is building data pipelines to take messy data from a variety of sources ranging from modern APIs to crappy legacy systems. We clean it up and combine them all into one master data source with a pristine, easy-to-query schema. We sell this to businesses, who all have the same basic building blocks: customers, products, subscriptions, etc. The systems they use to track these things are completely disjoint and often impenetrable for non-technical users (e.g., the CEO). We’re building a universal system for organizing and making available all of a business’ information.
You will be one of the first employees, working directly with the founders (former YC founder and the founding engineer at MongoDB). We have also raised our first round of institutional financing and are well-capitalized for the next several years.