Made some more progress on LDP-1450 text overlay

I tried the trick of expanding the source texture to twice its normal twice and enabling filtering.  This looks really good but there are still a few artifacts when the grey background is enabled.

Here are some screenshots for comparison:

Finally generating LDP-1450 characters in GLES2

I finally broke down and wrote some GLES2 code (including a new vertex shader) to render LDP-1450 text overlay.  While this may sound like a trivial task, it was actually quite difficult coming up with an approach that would run fast on the Raspberry Pi.  Trying to just render to a frame buffer directly would've been far too slow so I needed to come up with a tile system.  I am still making speed improvements, but I am pretty confident that I am going in the right direction.

Here is how the original LDP-1450 text looks:

 Here is how my generated text looks using no filtering (ie nearest neighbor):

Here is how my generated text looks using filtering:

This isn't an exact comparison because the LDP-1450 text I captured is in "grey border" mode which I have not yet tackled, so you will kinda have to imagine that the grey border is not there.

Anyway, my conclusion is that the filtered mode is too smooth and the nearest neighbor mode is too rough.  I think maybe if I increase the size of the source texture by a factor of 2 (or maybe more if necessary) that I should be able to use filtering and still see some pixel edges like on the original LDP-1450.  Stay tuned...

Worked on LDP-1450 text overlay some more

This morning I worked on the OpenGLESv2 side of things, writing code to make LDP-1450 text overlay run fast on the Raspberry Pi.  I am basically caching every character that can be displayed as a texture and then when I render it, each rendered character will be a rectangle with a portion of the texture applied.  It should be pretty fast.  If it's not... *ominous music*

Unfortunately, anything to do with GLES2 is quite difficult to get working right.  So this could take me a while.  I've been dreading it but I finally forced myself to do it so the train is moving.

Dexter status spreadsheet

For those wanting a larger overview of where Dexter is at, I made a spreadsheet showing what is finished and what is left to do.

You can find it here.

Added more commands to LDP-1450 code

This morning I added interpreters for most of the LDP-1450 text overlay commands, including unit tests.  The only one I haven't done yet is the command to actually change the contents of the text buffer.

Dexter update: Sony LDP-1000A probably won't make the cut

As you may have noticed, I have been exploring various video capturing technologies lately.  Part of the reason is to see if we can have more control over the video output signal than what the Raspberry Pi gives us.  Why?  Because of two reasons:
1) the Raspberry Pi does not let us control the VBI data that contains the frame numbers
2) the Raspberry Pi does not let us control when vsync occurs

Reason #1 is important because some players (PR-8210 and PR-8210A) are essentially output-only players and do not give the game any feedback as to whether they are functioning correctly.  In order to know whether the player is working, the game hardware needs to read the frame number from the VBI data.  So the fact that the Raspberry Pi does not let us control this VBI data was problematic.  However, I believe that we have discovered a way (and we will pursue it) to inject VBI data into the video signal in real-time (which is very cool).  So things are looking good for the PR-8210 games (Cliff Hanger, Goal to Go, MACH 3, and Us Vs Them off the top of my head).  Things are looking fairly good for Star Rider (PR-8210A) but it is still way too early to say with confidence that Dexter will definitely work with Star Rider.  But I am feeling good about the other four games at this point.

As for Reason #2, it turns out that the Data East games which use the Sony LDP-1000A (Bega's Battle, Cobra Command, Road Blaster) tell the laserdisc player when vertical sync occurs (unless I am very much mistaken) which is backward from how every other game works (the laserdisc player tells the game when vsync occurs).  I really am mystified how the laserdisc player can let an external source dictate when vsync occurs.  But regardless, this is something that I am quite sure the Raspberry Pi _cannot_ do (allow an external source to dictate vsync).  So the solution is for us to control the video signal more directly (ie through a DAC).  But at this point, I feel that going down this road would extend the release date of Dexter by a year at least, probably two to be realistic.  And I don't want to do that.  So I am hereby saying that as of now, Sony LDP-1000A support in Dexter v1.0 is out due to the constraints of the current design (ie using the Raspberry Pi).  I do eventually want to support this player, but I also want to ship a product!

That's the bad news.  The good news is that Sony LDP-1450 support is very much in and should be very good.  And the command protocol is almost the same between the 1450 and the 1000A.  So Dexter may still be able to be used with a Data East game but the game would require some hardware modifications (beyond my knowledge).

New SD card benchmarks

Well, before I go to a bunch more work only to find out that I can't succeed, I decided to try to do a sustained write to the SD card that would last about 10 seconds.  Therefore, I choose to write 300,000 512-byte blocks (about 146.5 MiB).


With a 12-bit 15 MHz Analog->Digital Converter, we will need to read in 180 million bits per second, or 21.46 MiB/second.  With two cards splitting the load, each of them would need to be able to write at 10.73 MiB/second.

From start to finish, it took 13.467 seconds, or 10.88 MiB/s which is too close to the minimum to be reliable.  Future runs were slower.  For example one was as bad as 14.59 seconds (10.04 MiB/s).  Another test yielded 14.06 seconds (10.42 MiB/s).

Without using the SD card's pre-erase feature (which I can only use for the first 4 gigs on the SD card for some strange reason), I am getting 15.06 seconds (9.72 MiB/s).

EDIT : with an optimization and no pre-erase I am now getting 14.2 seconds (10.3 MiB/s).

EDIT #2 : I just tried doubling the data written.  Now I am getting 9.83 MiB/s.

NOTE : It looks like the reason for these speeds is because I am using this kind of SD card.  I've found that bigger (physically) and smaller (capacity) cards can write faster.

Conclusion: Ok, so two SD cards are almost fast enough but not quite.  Incredible.  I was so close!  I'm so close now that I will probably just buy another SD card.  But dang.. 3 SD cards!  This project is broadening into gargantuan proportions.

Soldered the other SD card adapter

I bought 2 SD cards with adapters and this morning I soldered the other one.  I did a much better job this time. :)

SD card write benchmark complete!

Wow, I got this working a lot faster than I thought I would.  It's nice to not have too many problems to fix once in a while when writing new code.

I was able to write 60,000 512-byte blocks in 1700 milliseconds.

That means I wrote 29.30 MiB in 1.7 seconds, for a rate of 17.233 MiB/s.  That is just _barely_ fast enough to handle an 8-bit 15 MHz ADC.  Since the one I want to use is 12 bits, I will need to use two SD cards in parallel.  But I was expecting this which is why I purchased a second SD card at the same time I bought this first one! :)

I just hope this 17 MiB/s can be sustained over 30 minutes.  I kinda doubt it can but who knows?

Got SD card writing at 50 MHz!

I am now successfully writing a 512 byte block to the SD card at 50 MHz!  It's been very difficult getting to this point.

My next step is to try to write 60,000 blocks in a row (about 30 megabytes) and benchmark how fast the card can actually do it.  This will tell me how many of these I will need to run in parallel to do laserdisc video capturing.

VHDL shifters (reference)

This post is just for my personal reference later and likely will be of no interest to anyone.

I am writing this here because I consistently mess these up.

To shift a logic vector left (throwing away top bit):

counter(15 downto 0) <= counter(14 downto 0) & '0';

To shift a logic vector right (throwing away bottom bit):

counter(15 downto 0) <= '0' & counter(15 downto 1);

Successfully wrote to SD card!

I've finally worked out enough of the issues in my VHDL code that I've been able to successfully write 512 bytes to the SD card.

Here is how the correct CRC and response now look.

I've also fixed the way that I output to the CMD and DATA lines so that it is consistent with the way that the SD card does it (ie the correct way).

Notice that I now get the response of 0 + 010 + 1 where 010 means "data accepted".  Woohoo!

The mysterious SD card "CRC status response"

So I tried writing to the SD card.  I tried writing a continual byte value of 0xA5 (10100101b) so that the bits on data line 0 and data line 1 would alternate with each other.

A quick visual check showed that I was capturing this properly.

But wait.. what is that schlop right after the CRC on data line 0??  I was expecting the line to go low (busy) and then high again.  But it looks like there is another type of response.

A scouring of the SD card simplified specs revealed to me that after write operations, the SD card sends a "CRC status response" followed by a "busy" signal (holding the line low while it is busy writing).  The trouble is, the specs do not seem to define this response anywhere!  They just say unhelpful remarks like "see below" or "see elsewhere.

I finally decided to try my luck inside the SPI section of the specs, even though I am not using SPI mode.  And finally, in section 7.3.3.1, I found a section entitled "Data Response Token."  This does indeed seem to be what I am looking for.

Here is how it describes this token:

bit:
4 - always 0
3 - status
2 - status
1 - status
0 - always 1

010 - data accepted
101 - data rejected due to CRC error
110 - data rejected due to write error

As you can (hopefully) see in my screenshot, I am indeed getting the following bits: 01011, which corresponds with "CRC error."  And following that, I am seeing the data line being held low to indicate "busy".

So now, once I debug my CRC calculator defect, I should see 00101 (data accepted) followed by the line being held low (busy) while it writes.

On to debugging my CRC algorithm! (it works in my C code, just not VHDL!)

SD card reading

I've had some problems writing to the SD card so I decided to take a step back and try to get reading working first.  The hard part is verifying what exactly the "data address" in the spec refers to, how many bits are sent on each data line, the timing, and the CRC algorithm.  When trying to write, I was making a few assumptions about this.  But now that I am reading, I can verify it for sure.

First, I booted into linux and created a 512 byte block of repeating bytes with the value of "0xF0".  I then used "dd" to write this block onto the beginning of the SD card (thus wiping out the existing FAT partition information that was on there).  This created a bit pattern at the beginning of the disk that I would be able to recognize when reading (due to the way the bits are streamed out when in wide bus mode, this would create an alternating pattern of 1's and 0's).

Next came the challenge of being able to see what the heck the SD card was spitting out.  Trying to log in VHDL is a huge pain.  I had ordered a logic analyzer but it hadn't arrived yet.  Then I had an idea!  Since I was in control of how fast the SD card's clock pulsed at, I could slow it down enough that I could use an AVR microcontroller to log the data and spit it out to a serial port.

After a few calculations, I determined I would need to crank the clock down to about 1000 Hz and set the AVR to transmit at 230400 bps over the serial port.  I ended up using an ATMega 328p which worked great as I was able to run it in 3.3V mode.

Here is the resulting screen shot right after I send the read data command to the SD card:

As one can see, I am definitely getting the alternating pattern of 1's and 0's that I was hoping for.  PHEW!  This means that data address 0 does refer to the beginning of the SD card (which is what I was expecting but you never know for sure until you verify).

Also interesting is that the SD card starts sending data before it has finished responding to the read command.  That's weird.  But I guess the specs did imply that that was what would happen.

Next I need to verify how many bits are being sent on each data line (I am assuming 128* since there are 4 lines of which I am only capturing 2).  Finally I will need to see what CRC it is sending back to me and check it against my own algorithm that I think is correct.

After that, I _should_ be able to write something, dangit! :)

* - EDIT: Well, I obviously wasn't thinking straight.  The correct answer is 1024 bits on each data line.  This is because 512 bytes are sent and there are 4 bits sent per clock.  This should've been obvious *sheepish look*

Making SD card progress!

I've made more progress on talking to the SD card.  I've now completely finished the initialization sequence and am in transfer mode and ready to start multi-block writing.  That will be my next task when I work on this again.

I've also beefed up the logger.  It is really inefficient but still meets my timing constraints so I am pleased.  Here is what the logger spits out.  My comments are in italics.

- Reset

C00 go idle

C08 notify SD card that we're using 3.3V and we can support newer commands

C55 get ready to send ACMD41

A41 initialize

C55 initialization is busy, so we keep looping until it's ready

A41

C55

A41

C55

A41

C55

A41

C55

A41

C55

A41

C55

A41

C55

A41

C02 get SD card's CID

C03 get SD card's relative address

C07 select this SD card for transfer mode

C13 get current status

C55 get ready to send ACMD6

A06 switch into 4-bit (wide) data bus instead of 1-bit data bus

C06 switch from 25 MHz to 50 MHz (ie "high speed" mode)

SD card timing diagram

YES!!! I got a response back from the SD card!!!!

What I did basically was:

-- wait 128 cycles
-- send 400000000095 (CMD0)
-- send 48000001AA87 (CMD8, requesting 3.3V, check pattern of AA, CRC)
-- received 08000001AA13 (R7, 3.3V accepted, check pattern of AA echoed back, CRC)

Here is some data they don't tell you in the simplified SD card specs.

The relationship between the SD clock and the I/O is shown in this diagram.  I found it quite easy to have an internal clock running at twice the speed of the SD clock in order to achieve this timing.

The main thing to note is that the I/O (ie what you see on the DAT0 line) always changes on the falling edge of the SD clock and is always stable (so it can be read) on the rising edge of the SD clock.  Maybe this is obvious to some, but I agonized over it which is why I am posting it here.

Also, I am pretty sure that the data lines (including CMD) have pull-up resistors on them although I have not had this confirmed.

Also (thanks to Dave Spicer for telling me about this), you need to wait up to 128 clock cycles (may be less, I'd need to experiment) after powering up the SD card before you can send the first command (CMD0) to it.

Time to try to write to an SD card

It's time to learn how to write to an SD card.  This is so I can capture raw laserdisc video and process it on a PC.

I know it looks like the left two pins are soldered together, but multiple continuity tests show otherwise.  I am not going to try to resolder these unless I absolutely have to (ie getting corrupt data).

Color burst zoomed in

Here is an oscilloscope view of the color burst before (green) and after (yellow) it has gone through the ADC/DAC.

It's interesting, it appears that you can see the discrete binary transitions on the DAC output. I have to admit, I thought that there would be a bit more resolution to those binary waves. Parsing that and trying to find the actual clock cadence may be tricky in software. (tricky but not impossible)

I got the ADC/DAC working!!!

It works!!!

First of all, I rigged up two separate 3.3V power supplies (using a common 5V supply which uses a common 9V supply).  I don't know if this is a good idea or not but it's what I decided to do.  If anyone sees this as a debacle, let me know :)

Then after some experimenting, I finally got the DAC outputting something similar to the analog source.

Just a black screen:

And here it is with video playing:

The next step is to figure out how to stream this out to dual SD cards at 30 megabytes per second! :)

The ADC/DAC has arrived!!

I went to check my mail around noon today and was disappointed to find only envelopes in the "mailbox" (I don't know what you call these things).

But then as I was walking up my front steps, I found this!

Ohohoho...

 What the... what's in this black box?

*drool*

Oh I see what was in the black box.  They threw in 5 SMT ADC's.  What the heck do they think I am going to do with these? :)

BatchPCB turnaround time (again)

Back in December, I wrote:

"For those curious, I ordered my latest batch of PCB's from BatchPCB on 13 Dec 2012 and they shipped them to me on 31 Dec 2012.  So 18 days not including shipping.  Also note that this was during the holidays and I ordered at the end of the week so it could be faster next time."

And now I write again.  I placed my BatchPCB order on Jan 15th, 2013 and I just received notification that it shipped just now.  That is 17 days unless I am very much mistaken.