You da man, Bortels!

Probably worth recording here methods of getting code onto the iPad as well (once it has been decoded from its codea format, if need be).

For linux users, libimobiledevice from git (probably not in any distribution) works just fine and can mount the Codea file directory as read-write. Then files can be copied to and from with impunity. In particular, this method does not require jailbreaking.

If you use windows or a mac, the iExplorer program at http://www.macroplant.com/iexplorer/ is free, and works fairly well, and also doesn't require jailbreaking. The access is tethered, meaning you need to be plugged into the ipad for it to work. (the libimobiledevice Andrew mentions above is also a tethered solution).

In all cases of moving files (rather than cutting/pasting), you'll need the contents of the .codea file broken out and local.

That's a very clean workaround. Much appreciated.

Looking very good Mr. Bortels. Very good indeed. Many thanks.

Here's how I get code into and out of Codea via Linux: http://natpryce.com/articles/000792.html

Nat, thanks for that. It's very close to what I do. I spotted a couple of improvements on what I do, and for some reason I don't have to use sudo so I'll take a look to see what it is that means that I don't have to.

(Actually, here's a first guess. Is your user account in the fuse group?)

But most of all, I want to know what the GrumpyPigs project is!

Update: Ok, I had some unexpected decoding issues (spurious null characters in some but not all files, sigh) - I think it's 'fixed'.

See http://home.bortels.us/decodea/results/

The next step is upload. Tomorrow (well - later today, it's 1:30 am here) probably. Good opportunity to update my web upload mojo, there are nicer ways today than the old file uploader dialog box (maybe a drag and drop thingamabob, just for fun).

I'm also going to make a newer decoder available shortly - the final version I'm using on the site itself. It creates both the html pages you can see on the site, but also a .zip version that we can share that should be a bit more friendly for the archaeologists trying to figure out this stuff (probably ME) 20 years from now.

I'm counting on this for a new version of Spritely. I've got a pop-up menu for copy vs. rename, a bug fix on the keyboard (didn't notice until yesterday that if you typed between keys the keyboard was "transparent" and was passing touches on to other elements), and a simple "undo" function that at least allows you to recover when you do an "oops, I didn't meant to press clear."

But the code sprawls across enough classes that without projects, it'll be a pain. Oh, and I guess I'll need to pull the images that were stored in project data and put them in code, so the code will just get bigger and sloppier.

Man, I hope Apple sees reason on allowing projects to return.

So - two things (both mainly questions aimed at TLL):

1) @Simeon - you might also simply use the "code is speech" argument. It's fairly well established, at least in the US, that computer source code, which is what we're dealing with, is a form of speech - and Apple is a US company. Now - they're private, so they can restrict what they want in their walled garden - but the .codea files don't exist in their garden, by definition. What Apple is doing here is preventing their end-users from engaging in free speech among themselves - they are practicing censorship, of a most egregious sort, because they are banning an entire class of communication based on what they think someone might say. (it's also ineffective - we'll say it, just elsewhere) This is what China and Syria do, and sadly this is what SOPA is trying to do (if it passes, it will surely be struck down as unconstitutional, but until it's harmed many...)

2) There has been some talk about tab folding - is it the case that I could, for example, take the Spritely many-tab source code and just paste it all into a single "Main" tab and expect it to work? If so - I can do that in the .codea extractor, so an end-user should have at most one cut/paste operation to import any project? (I'm ignoring possible file size limits - I wonder how big the cut buffer is...)

Mark, maybe it's time to get serious about versioning. If you make changes, perhaps you could indicate which files have changed so that we only have to download those ones.

I did start versioning the individual classes with Spritely 0.80 -- and updated the versions properly in 0.90... but since then I kind of lost track as I was fiddling here and there.

I think I'm declaring the whole mess 1.0 on the next release, then I'll try to be good from that point forward.

Guess I will have to wait until 1.01 then, because the .0 version never works right! :-D

Not that it's there now on the latest, but versioning individual classes won't help much when .codea imports are monolithic. If we kept versioning, than an update merge might make sense.

A General tipp - if i Write a Game, i always insert a "new blank file". I call it "gameInfo", and i write in:

• my name

-Name of the game

• the version of the game

• whats new in this version

• previos versions

Something I'd like to see - if line 1 of Main is "-- something" - something is a short description to display below the icon in the main Codea view. And - give us a setContext() way to set the icon, please. :-)

Ok - I think it's ready to go. Or - it's not, but I want someone else to break it.

The page is http://home.bortels.us/decodea

You can go there, upload codea files. They'll get magically decoded (once per minute), and both a .zip of the archive and a nice html file that's safari mobile copy/paste friendly will be created. So - if you have 1.2.7 and can't eat up a tasty .codea file - you can go here for the next best thing, for now.

Go to http://home.bortels.us/decodea/results for all of the results.

Will this clobber my bandwidth? Perhaps. But I'll leave it up for the time being, and we'll see what happens.

This is a "least common denominator" solution - once it works and seems to be stable for the community, I (and others, no doubt) will begin to work on local .codea importers - idea being, I'm jailbroken, so I should just be able to download a .codea file, double-click on it, and have it magically show up on my ipad.

Apple forcing the removal of .codea import does not actually stop us from importing .codea files - it just makes it inconvenient for everyone, and forces us to work around their rules, making the competion more attractive by comparison. At some point, working around their rules will become the norm for most users - that'll be interesting. As a general point for Apple to consider, making unenforceable rules is almost always a bad idea.

PS. The page is almost all open source stuff, and the part I wrote I release CC0 - I'll make an archive of the workings available later on, for anyone who wants to host their own decoder, in case I get crushed under the load. (If you have a codea file you want to decode - check the results page first to see if it isn't already there!)

Very cool @Bortels. Love the name, and the output is really easy to copy from.

Very nice site.

Bortels, what does it do with the data file btw? Just ignores them? How about having data.plist in the zip file at least? (sorry about that i havent played with it yet, still hanging on to 1.2.5 until i export everything)

Right now, it just ignores them - there's no way to add them in to Codea by hand, so there's not much we can do with it.

Hmm - that'll be where the private project persistent data is located, isn't it. That binds things a bit.

I can put it into the .zip file that's also created, or simply add it to the files displayed - but without a way to get it into the project on the ipad - not much use.

Let me ponder...

I guess I could make a "data loader" that populates persistent data, and have it take the relevant content from Info.plist and load it - to make proper use of it, the project author would basically have to check if their content was in persistent storage, then call "reload_plist()" or such if it isn't nil. Does that make sense?

Yes - this is fixable, perhaps.

I was thinking i'd use the zip file to upload to the ipad with iexplorer

You idea is interesting too, afterall the current storage is all text.

Well - I can certainly expose the plist file - I'll do that shortly. Yes - that will help those not doing copy and paste.

And... done. The Info.plist shows up in the zip, as well as visible in the html if someone wants it.

And I think I have a cunning plan. I... nope, it won't work.

@Simeon, we need to be able to create tabs programatically. Please. Pretty Please.

(If it's not obvious, the idea is to make a "loader" that has all of the tabs, plus the persistent data, in it - copy/paste the single loader in, run it, it creates all of the other tabs, populates the persistent data, then... hmm, how could it replace itself.... more pondering to do...)

-- pseudo-code for the win

loader=true
loader-Main = "the Main tab, encoded (prolly base 64)"
loader-Other = "A tab named Other, same deal"
loader-Whatever = "Whatever tab..."
function setup()
   if loader then
      -- we make the tab, decode "loader-tabname to it"
   end
   -- now swap out the original main tab (yes, this could be weird)
   renametab("Main", "loader")
   renametab("Main-new", "Main")
   removetab("loader")
end

Just thinking out loud. Apple can prevent you from importing code, but they can't prevent you from doing things that make it easier for us. :-)

Interesting thought. Might as well allow programmatic creation of projects, too. I'll look at how we can do it.

That would be even better - you just grab and cut and paste, and the temporary project you just did recreates the whole project.

Heck - I can see uses for that aside from restoring projects...

Um... I like the idea, but it could crash the iPad and fill up Codea's catch if it was put in a loop. Just my though.

Nah - only if the loader got too big, and even then it should only make you need to restart codea. Plus, Simeon and crew are looking at the big file issues.

What if we set up like the file server (on apps like file app, quickoffice, etc) that transfers wirelessly over an http://(ip address here)? It would allow for easy transfer to commuters for projects and maybe be a loophole, for we would be downloading the projects to our computer an not the iPad itself. Though we would still be executing them there...

Apple isn't going to allow us to directly put code onto the ipad for Codea, not unless appeals work, not without actually using paste, and I'm surprised they allow that. None of the other tricky ways to get code there are likely to be approved.

Point being, we have the appeals process, or we have to learn to live with copy/paste, and then work around the issue. And - work around we can. It's just ugly, and stupid.

Update: I am now also copying over the uploaded .codea file to the results directory for reference. So - it's a bit of a one-stop shop for grabbing .codea packages too, which is nice.

Code Authors: I'd recommend, in addition to what you're already doing, you make a nice versioned copy of your .codea file, and upload it. Inside the main, as the first chunk, I'd suggest putting your name, home website if any, and any licensing you want to include. If you keep it to a nice 7ish lines, it should display nicely in the top of the file copy box.

If someone wants it, I'm happy to search uploaded codea files for some special tag that means "please don't put this up or allow it to be decoded, bortels, you busybody!", and remove those from processing... I suggest a single line with "-- NO DISTRIBUTION". Reasonable?

I am also thinking it might be wise to simply reject uploads that are already there, to prevent vandalism. Not that I think we'll see it, but this is a case where someone can upload stuff at will - better to prevent it now than correct it later. I'm not too worried about spammers, because they'd have to figure out how to export a valid .codea file, but meh.

I've taken the liberty to grab and decode some of the projects I've been using and admire, mainly to confirm there aren't more bugs lurking in my decoder. So - some of you may see your projects already there. I'd really like to see the others that have been posted here, if only to confirm they decode ok as well.

FWIW - turns out there's a Data.plist with the local persistent data. The problem of how to get that data into a cut-and-pasteable form remains. Or rather - it's easy to find, and select - and there's no good way to paste it in because we don't have editor access to the .plist files.

TLL - maybe that's the easy solution. Just let us edit Data.plist as well as the lua files. Yes, it's XML and it's ugly, but it's not that bad. And it'd be easy to fix issues - just kill the tab.

Just looked at my ones. A couple of things strike me:

1. The script should ignore files that aren't .plist or .lua. I see a few .lua~ in my projects that I'd not realised where there. There's no real danger in having these in there as Codea just ignores them, but a user who didn't know that might go to the bother of trying to cut-and-paste them without realising that they shouldn't (or couldn't).
2. As I've upgraded, I can no longer generate the .codea files. In particular, I can't go back and insert any key fields such as author or original url. I'm not so bothered about the attribution, but I do update my projects from time to time so it would have been useful to have a "Get the most up to date version from ...". I know that you've just used these to test the idea, but if you are going to harvest .codea files en masse (which is absolutely fine by me as far as mine are concerned) then a link to where you got it from would be nice.

1. ok - I'll have to swap the logic a bit, the script currently does "all but..." and I'll need to make it do "none but...". No worries.

2. That's cool - but the "where I got it from" is "somebody uploaded it" - I don't force people to provide a source, they may not know. So - how do we let authors who have upgraded and can't make .codea files update? Nasty question - there's a lot of "junk dna" in the .codea files, I can't make one, without just using the ipad and Codea. Let me ponder this today - we'll either find a way for non-.codea-generating authors to make a .codea file (probably by uploading a .zip of the project), or... dunno. Something. I mean - aside from this, if you update your projects in general, you still have a stale .codea file on your site, right? I was hoping to avoid making them, but we may have no choice.

I'm not clear as to your goals for this project. Is it to host all (publicly available) Codea code? Or is it just as a way for people who have upgraded to import .codea files? If the latter, then there's no reason for making new codea files for you to extract the data again from! Yes, I have stale codea files on my site but as I update projects then I'll delete them because the project will be the newer one. Or I might say "If you haven't upgraded to 1.2.7 then you can get an old version as a .codea file".

Incidentally, I'd be more than happy for someone to have a central place where we can get codea code from. I'll continue hosting my stuff on my website, but I've no issues with someone harvesting them. As I say there, anything that originated with me is in the public domain (or CC0). It's just that to be as useful as possible, it's good to have links back to my website from my code so that people know where to get the latest version (should they want to! I'm under no illusions as to the popularity of my code).

The goal is to have a way for a user to get to .codea file contents without having to install something local (like perl). A web interface does that.

An immediate sub-goal was to throw a bunch of .codea projects at it, to make sure they all decode properly, to vette my decoder; there is no better proof of correctness than to confirm it's actually working on as many known files as I can find. (If I had a good spec for the file format, yes, I could do unit testing and be more sure - but I don't.)

I am also of the belief that there will be more than a few people who refuse to downgrade to 1.2.7 and lose the ability to import .codea files - So it's nice to have new, fresh .codea files around for those users as well. (Ha! Won't downgrade??? Well, too bad - you get to cut and paste anyway if you want the latest stuff...) It would be awesome to be able to upload a .zip file, and have it made available in the same manner, including as a one-click install .codea file for those who have yet to downgrade.

This assumes, of course, that it can even work - if you use a 1.2.7 feature, it would break even if imported as a .codea file. Caveat etc etc.

The Uber-goal is to demonstrate in a concrete manner that Apple removing code import via .codea files doesn't actually accomplish their "no importing code" goal; it simply makes the user experience suck more, and the blame for that is solely at their feet. I hope that at some point someone there raises their head out of the sand and says "Uh, guys - Gruber is saying we suck. On the Internet. No, really - Gruber. Yes, that Gruber. I know he likes our stuff - maybe we're doing the wrong thing here after all?"

I can dream.

PS. My intent is to take the whole site, once I'm sure it's mainly stable, make a nice zip file, and make it available for anyone who wants it, so the whole world isn't dependent on my home DSL. It depends on apache, php, and perl - all of which are stock on any ubuntu installation. It could be made to run with little difficulty on other platforms, including cloud (it's got options for google's cloud). I'll probably post it later today, barring fires.

The decoder will now only include .lua and .plist files, so people going directly to the directory via various means won't have their .git and .lua~ and so on included in the output here.

The last line in the FAQ on the decodea site has a link to a copy of the entire site. It's less than 400k, including all of the .codea files I've put in so far. Just for the record. If you grab it and see a giant security hole staring me in the face - let me know.

Hail Eris... Nice

I am chaos.

I am the substance from which your artists and scientists build rhythms.

I am the spirit with which your children and clowns laugh in happy anarchy.

I am chaos.

I am alive, and I tell you that you are free.

I like the idea of copy and paste a single block of source into a new project tab with some kind of embedded instructions/comments that tell Codea to create new tabs.

Would it be possible for Codea to have its own built-in browser to import .codea or .xml or .txt files into a new project or is that against the rules too? I've seen a few apps that do that kind of thing e.g. Modizer which will import/download old mod/tracker style song files (from old Amiga/ST/console games usually) and play them.

Simeon/TLL - thank you for a fantastic iPad app!

@mpw1966uk an embedded browser is a good idea. But I think anything we do regarding importing will have to be vetted by Apple at this point. If we try to include anything without being up front about it they are likely to remove the app.

I keep getting a message from the iExplorer telling me I have used up my 5 browses for this session. Is there a way to turn this off?

I'd rather people grabbed my code from the GitHub repo than from an unpacked old version. Maybe a bookmarklet could copy the code to the clipboard so it's easier to copy and paste it on the iPad.

@Nat - You might as well just select the code, copy it, and then put it in each correct tab. If TLL made a feature that lets you copy a chunk of code and put it into Codea with all the code already organized into tabs, that's technically importing.

@Bortels: did I understand that properly that upload of code does not work from iPad?

@CrazyEd - It wouldn't work on an iPad with their lack of a file system.

Ah, yes, thought about something like this... Thanks for clarification, wasn't sure about that

I also just tried it with Phone Drive and that doesn't allow it either.

@crazyed - you cant upload directly to the page from the iPad - you'd need to email to posterous or such, then use a pc to upload it to the decoder.

TLL - I'm unclear, can we export projects still???