phantomraspberr Contributor - Level 2 2018-12-04

2018-12-04

Debugging a bricked Ditto X4

Disclaimer
I am in no way affiliated with TC Electronics or anyone/anything to do with them. I'm just an annoyed customer with a bricked Ditto X4 and some technical know-how.

Edit: The Solution!
This issue is now resolved, so instead of having you read pages of write-up, I'll condense the solution for you here. If your pedal won't start up, showing either a single red LED above the LOOP1 button or all orange lights, this will get you back in working condition.

Note #1: This will erase any loops you have saved! If you want your data you'll need to copy it off the SD card first and copy it back on once you're finished re-imaging the card.

Note #2: This will void your warranty! If your pedal is still under warranty I suggest you explore warranty options first, DIY last. Having got that out of the way...


[list=1]

  • Make sure you have an SD card reader/writer on your computer. For example: https://www.amazon.com/Anker-Portable-Reader-RS-MMC-Micro/dp/B006T9B6R2/ref=sr_1_4?s=pc&ie=UTF8&qid=1544220198&sr=1-4&keywords=sd+card+reader

  • Download Etcher: https://www.balena.io/etcher/

  • Download the X4 SD card image (v1.3.00 build 89): https://mega.nz/#!NMdRiAIY!GtrYevDAjHzPgiYAiwUaj_X9qjVVJ-HF3ZX1MP1mrLo

  • Remove the SD card from your X4 by removing the bottom of the X4, slide the SD card clip to the side to release the catch, then lift the card out. You'll need a torx bit for your screwdriver in order to open the pedal.

  • Put the SD card into the reader/writer.

  • Run Etcher and select the X4 image you downloaded earlier.

  • Write it to the SD card using Etcher - it's self-explanatory.

  • Once it's finished, put the SD card back into the pedal, put the back on, and power it up to test.

  • Apply the latest firmware update from TC.

  • Play music!




  • Forum user fzzbx said that Etcher didn't work for him, but Win32DiskImager did, so here's his instructions for making it work:

    fzzbx wrote:

    My solution was to use a program called Win32DiskImager instead.

    https://sourceforge.net/projects/win32diskimager/

    I renamed the .bin file provided by phantom to a .img file, and used this program to image the SD card.

    It worked on the first try!


    fzzbx also makes a good point that your SD card could have gone bad, which means a new image won't work and will probably fail to write to the card - you may also need a new SD card. This one would be fine: https://www.amazon.com/SanDisk%C2%AE-microSDHCTM-8GB-Memory-Card/dp/B0012Y2LLE/ref=sr_1_4?ie=UTF8&qid=1544302078&sr=8-4&keywords=8gb+micro+sd+card


    Original post continues here.



    Symptoms
    When power is applied, all the lights come on in red and green (people call it orange?) A few seconds later all the LEDs go off and the single solid red LED at LOOP1 comes on and stays on. It doesn't respond to button presses.

    If I put a blank SD-card in the X4 instead of the one it came with, it never boots past all of the lights coming on - it doesn't get as far as turning on the LOOP1 red LED.

    Boot Mode
    You've all read the instructions to get it into boot mode. They don't work for this X4. Powering on with the LOOP1 button down does nothing.

    Firmware Update to 1.3.00r89
    It fails. The X4 is detected by the Windows Ditto X4 updater as a "Microsoft Wavetable GS Synth", which I believe is a MIDI device. I guess that means it's not in boot mode, but I tried installing the update regardless. It gets all the way to 100%, then stalls and does nothing. No lights change on the X4, the updater program just sits at 100%.

    TC's Technical Support
    Turns out they're neither technical nor supportive. I filed a ticket with a detailed bug report of the problem, but they just referred me to a repair shop in Texas and closed the support ticket. I tried repeatedly to get more technical information about debugging the issue, but didn't get anywhere. Like most of the TC support encounters I've read about, it wasn't helpful.

    DIY
    So here we are. A bricked X4 and a small workshop full of electronic toys. Here's tonight's efforts:


    (https://i.imgur.com/8c61Cgf.jpg)

    Architecture
    It turns out to be powered by a small NXP i.MX283 ARM9 system-on-chip. There's an ATMEL ATtiny48 microcontroller, too. The ISP header is exposed but unpopulated.

    My guess (to be confirmed) is that IC1 (64k serial EEPROM) contains a boot loader that's executed by the MX283. The boot loader (U-Boot) presumably boots a Linux kernel off the SD-card, which loads the pedal's control software. This is speculation at present.

    I don't know what the ATtiny is doing. I attached an ISP programmer to it (my trusty TNM5000):



    It detected the ATtiny without any problem:



    I checked the fuses on the ATtiny and it's been locked, making it impossible (ok, infeasible) to dump the firmware out of it:



    There may be an opportunity to use the bootloader to dump the ATtiny ROM, but for now that's out of scope for what I need in order to diagnose the problem.

    Debugging Interfaces
    There are at least 3 UART debug interfaces onboard.
    J15 (not marked on the image above) is a read-only serial debug log at 115200 8N1
    J11 is labeled "debug comm port". I haven't connected to it yet.
    J17 is a read-write serial console attached to the 1st-stage bootloader, U-Boot. 115200 8N1.

    J17 Serial Console
    I used a Sparkfun "beefy" serial/USB adaptor (https://www.sparkfun.com/products/13746) to hookup the RX, TX and GND pins at J17. This is what I get at power on:

    [code]
    U-Boot 2013.01 (Nov 16 2015 - 17:20:34)

    CPU: Freescale i.MX28 rev1.2 at 454 MHz
    BOOT: I2C #0, master, 3V3
    I2C: ready
    DRAM: 128 MiB
    MMC: MXS MMC: 0
    *** Warning - bad CRC, using default environment

    In: serial
    Out: serial
    Err: serial
    Net: FEC0 [PRIME]
    Warning: FEC0 using MAC address from net device
    , FEC1
    Warning: FEC1 using MAC address from net device

    Hit any key to stop autoboot: 0
    1024 .
    1024 ..
    12288 lost+found
    ** File not found /update.tar **
    Failed to mount ext2 filesystem...
    ** Unrecognized filesystem type **
    ext4fs_devread read outside partition 1073750502
    ext4fs_devread read outside partition 8520198
    Failed to mount ext2 filesystem...
    ** Unrecognized filesystem type **
    ext4fs_devread read outside partition 8520198
    Failed to mount ext2 filesystem...
    ** Unrecognized filesystem type **
    internal error
    TCE-X4 U-Boot >
    [/code]

    The environment shows more information about what's happening at boot time:


    boot_recovery=if ext2ls mmc 0:${partition_recovery}; then if $update_present; then if ext2load mmc 0:${partition_recovery} ${loadaddr} /boot/splash.update; then vl3lcd ${loadaddr} ${filesize}; fi; else if ext2load mmc 0:${partition_recovery} ${loadaddr} /boot/splash.recovery; then vl3lcd ${loadaddr} ${filesize}; fi; fi; mw.l $loadaddr 0; if ext2load mmc 0:${partition_recovery} ${loadaddr} /boot/uImage; then if iminfo $loadaddr; then setenv env_state ${env_state},STATE_BOOT_FROM_RECOVERY; setenv partition ${partition_recovery}; run bootargs_ext; bootm ${loadaddr}; exit; fi; else setenv env_error ${env_error},ERROR_RECOVERY_UIMAGE; fi; else setenv env_error ${env_error},ERROR_FS_MOUNT_RECOVERY; fi; echo 'internal error'; exit;
    boot_working=if ext2ls mmc 0:${partition_working}; then if ext2load mmc 0:${partition_recovery} ${loadaddr} /boot/splash.working; then vl3lcd ${loadaddr} ${filesize}; fi; mw.l $loadaddr 0; if ext2load mmc 0:${partition_working} ${loadaddr} /boot/uImage; then if iminfo $loadaddr; then setenv env_state ${env_state},STATE_BOOT_FROM_WORKING; setenv partition ${partition_working}; run bootargs_ext; bootm ${loadaddr}; exit; else setenv env_error ${env_error},ERROR_WORKING_UIMAGE; fi; else setenv env_error ${env_error},ERROR_WORKING_UIMAGE; fi; else setenv env_error ${env_error},ERROR_FS_MOUNT_WORKING; fi; x4led 0x80;run boot_recovery;
    bootargs_ext=setenv bootargs console=ttyAM0,${baudrate}n8 rootfstype=ext4 root=/dev/mmcblk0p${partition} rw rootwait ssp1 env_error=${env_error} env_state=${env_state}
    bootcmd=run bootcmd_vl3
    bootcmd_vl3=run init_scripts; run set_contrast; run mount_scratch; run test_recovery_button; run test_pot_calib_button; run test_update_present
    bootdelay=1
    bootfile=uImage
    env_error=,ERROR_FS_MOUNT_WORKING,ERROR_RECOVERY_UIMAGE
    eth1addr=00:04:00:00:00:01
    ethact=FEC0
    ethaddr=00:04:00:00:00:00
    ethprime=FEC0
    init_scripts=setenv env_error; setenv env_state; setenv linuxmode; setenv update_present false;
    loadaddr=0x42000000
    mount_scratch=if ext2ls mmc 0:${partition_scratch}; then true; else setenv env_error ${env_error},ERROR_FS_MOUNT_SCRATCH; false; fi
    partition_boot=1
    partition_extended=4
    partition_recovery=2
    partition_scratch=5
    partition_user=7
    partition_working=3
    set_contrast=mw.l 80040088 20000000; mw.l 80064008 c0000000; mw.l 80064010 00ff00e3; mw.l 80064020 001e00ff; mw.l 80064004 00000001;
    stderr=serial
    stdin=serial
    stdout=serial



    LED control
    From the environment we can see that the boot loader is using the command "x4led" to change the color of the LEDs in response to different boot events. For example, at the end of the "boot_working" script in the environment we can see the command "x4led 0x80", which turns the LOOP1 LED red. That's exactly the failure I'm seeing!

    I played with it a bit. Here's "x4led 0xfc" (the LEDs look green but they're actually orange):



    And here's "x4led 0x66":



    Partitions
    This is interesting:


    partition_boot=1
    partition_extended=4
    partition_recovery=2
    partition_scratch=5
    partition_user=7
    partition_working=3


    The boot loader expects to find the partition layout of the SD-card as shown. In Linux parlance, the "working" partition would be /dev/mmcblk0p3 and the "recovery" partition would be /dev/mmcblk0p2.

    Boot Failures
    Here's what happens when we try to run the "boot_working" script:


    TCE-X4 U-Boot > run boot_working
    Failed to mount ext2 filesystem...
    ** Unrecognized filesystem type **
    ext4fs_devread read outside partition 8389126
    Failed to mount ext2 filesystem...
    ** Unrecognized filesystem type **
    internal error
    TCE-X4 U-Boot >


    Here's the definition of "boot_working" made pretty:


    if ext2ls mmc 0:${partition_working}; then
    if ext2load mmc 0:${partition_recovery} ${loadaddr} /boot/splash.working; then
    vl3lcd ${loadaddr} ${filesize}
    fi

    mw.l $loadaddr 0

    if ext2load mmc 0:${partition_working} ${loadaddr} /boot/uImage; then
    if iminfo $loadaddr; then
    setenv env_state ${env_state},STATE_BOOT_FROM_WORKING
    setenv partition ${partition_working}
    run bootargs_ext
    bootm ${loadaddr}
    exit
    else
    setenv env_error ${env_error},ERROR_WORKING_UIMAGE
    fi
    else
    setenv env_error ${env_error},ERROR_WORKING_UIMAGE
    fi
    else
    setenv env_error ${env_error},ERROR_FS_MOUNT_WORKING
    fi

    x4led 0x80
    run boot_recovery


    So we can see that U-Boot starts, runs "boot_working", fails, sets the LOOP1 red LED, then calls "run boot_recovery", which also fails, causing U-Boot to drop to a command-line. We can also see the individual errors happening for first boot_working, then boot_recovery. Here's the error from boot_working:


    Failed to mount ext2 filesystem...
    ** Unrecognized filesystem type **
    ext4fs_devread read outside partition 8389126


    And here's the error from boot_recovery:

    Failed to mount ext2 filesystem...
    ** Unrecognized filesystem type **
    internal error


    Diagnosis
    It looks like there's nothing wrong with the pedal, which is great news. Instead, the problem seems to be that it can't boot from the SD-card due to errors mounting the working partition (mmcblk0p3) and the recovery partition (mmcblk0p2).

    Time to take a peek at that SD-card. To be continued.

    • 0 Kudos
    • 29 Replies
    • Reply
    Highlighted
    Contributor - Level 2

    Re: Debugging a bricked Ditto X4

    Awesome debugging job indeed Phantomraspberryblower! My compliments, if only I had read your request in time I would have send you the image file. My Ditto X4 is not bricked though. However, I’m experiencing a serieus delay when sending a Midi start command (especially on 1st command). I would welcome the possibility to e.g. Use dipswitch 4 to allow Start/Stop command instead of Record/Play/Dub.
    Also X4 sync-drifting in time is a big problem which many X4 users experience. I wonder if you face similar problems? I’m running build 91 too. Thanks ins advance, hoping your outstanding work will trigger the development team of TC-electronic.....
    PaulHolland
    Highlighted
    Contributor - Level 2

    Re: Debugging a bricked Ditto X4

    lyounkins wrote:
    phantomraspberryblower, I gotta say that's some excellent engineering work you did. Well done!


    Thank you!

    Mike@TC wrote:

    Hi all,

    everybody that experiences issues with his or her Ditto X4, please get in touch with our tech support teams via the link below.

    We have seen the described issue with some SD cards failing and the units can be fixed as part of service repair.

    ...

    If the LEDS are not turning green (and stay lit) - the SD card might have failed and you should get in touch with our support team to help you getting your unit fixed.

    best regards, Mike


    While I appreciate the input, it's very late and the first thing I did was to get in touch with support. They were so non-technical that when I asked if an SD card image was available for me to burn to a new card, I was sent a JPG of the Ditto's PCB. For real, that actually happened. I gave up and this entire thread is the result.

    The instructions I provided should be useable by all but the most technically averse pedal owner, for whom a support ticket and repair bill will remain the only feasible option. For everyone else, we now have an option that costs under $10 (or nothing if their SD card is fine) and half an hour of their time.

    PaulHolland wrote:
    Awesome debugging job indeed Phantomraspberryblower! My compliments, if only I had read your request in time I would have send you the image file. My Ditto X4 is not bricked though. However, I’m experiencing a serieus delay when sending a Midi start command (especially on 1st command). I would welcome the possibility to e.g. Use dipswitch 4 to allow Start/Stop command instead of Record/Play/Dub.
    Also X4 sync-drifting in time is a big problem which many X4 users experience. I wonder if you face similar problems? I’m running build 91 too. Thanks ins advance, hoping your outstanding work will trigger the development team of TC-electronic.....


    I haven't experienced those issues, but I'm not a midi user.

    Regarding DIP4, I would imagine that's entirely do-able if you're technical enough to dive into the userspace software running under Linux on the pedal. Good luck!
    Highlighted
    Contributor - Level 2

    Re: Debugging a bricked Ditto X4

    My pedal failed again today. This time it appeared to boot, going through various sequences of lights before eventually finishing with all green lights on while being completely unresponsive to any button presses. If I held down LOOP1 to get into flash mode, it worked - I could enter the updater just fine.

    So I whipped out the old SD card (the original one from the manufacturer) and swapped it for a freshly imaged new Samsung 16GB card I borrowed from a Raspberry Pi (using the instructions from the first post, above). I also added a little bit of foam on top of the SD card holder in the pedal so that as I put the lid back on it applied a little - but not overly much - pressure to keep the card snugly seated.

    It works perfectly again. It looks like the fix I identified is good for multiple failure scenarios.
    Highlighted
    Contributor - Level 2

    Re: Debugging a bricked Ditto X4

    Hey there...

    @phantomraspberr 

    @fzzbx 

    I was able to fix my pedal with both of your advice! I downloaded the build file and win32 program and it worked! You're a lifesaver! - Greg 

    Highlighted
    Contributor - Level 1

    Re: Debugging a bricked Ditto X4

    Hi everyone!

    @phantomraspberr and @fzzbx , thank you 3000 for your unbelievable work! I've found your post only this morning after months (years?) of desperate research, and my Ditto X4 eventually came back from the dead thanks to you two!

    You guys are fantastic!

    Highlighted
    Contributor - Level 1

    Re: Debugging a bricked Ditto X4

    Here's my experience of this problem and fix, for what it's worth.

    IT WORKED!!! My pedal is normal again. Thanks @phantomraspberr and @fzzbx. But it was a **bleep**fight...

    I found that my Ditto SD card, my SD Card holder that expands the micro card to fit my computer's reader, and my computer's card reader were all defective. It took a good long detective session with two computers and two SD Card holders to work through the problems until I could flash a new card.
    So if you're having difficulties, consider that you may have a case like mine with 3 or more defects all in a line.

    balenaEtcher looks nice, but it's a much larger file, served more slowly than win32diskimager. I live in Australia so my internet connection is rubbish. The difference between downloading balenaEtcher and win32diskimager was a full half hour.
    Also the disk image linked by the OP is almost 4GB - which takes 12 hours on my ADSL1 connection. If you can, consider finding a faster connection somewhere before downloading it. And once you've downloaded it, store it somewhere til Rupert Murdoch dies. It can't be that long now.

    Happy looping!
    Ballantyne

    Highlighted
    Contributor - Level 2

    Re: Debugging a bricked Ditto X4

    I deleted an earlier post saying the fix hadn't worked for me. Turns out I was just a dummy. Fix worked like a charm. My pedal is fully back in action!

     

    Thanks so much for sharing your knowledge @phantomraspberr and @fzzbx !

    I have had months (possibly over a year) of headaches with this pedal.

    Caleb Heineman
    Highlighted
    Contributor - Level 1

    Re: Debugging a bricked Ditto X4

    This worked perfectly for me to fix an out-of-warranty Ditto X4 bricked with all lights on. Thank you @phantomraspberr for your skill and perseverance in figuring out this solution!

    Highlighted
    Contributor - Level 1

    Re: Debugging a bricked Ditto X4

    What was final solution? I'm in the same boat.  I have bought 2 Micro SD cards 

    Tried etcher and Win. The ditto never boots holding loop1. 
    please advise. Thanks !!!

    my phone # is 312 909 5194
    Highlighted
    Contributor - Level 2

    Re: Debugging a bricked Ditto X4

    I was able to follow the above instructions and had success. After using etcher the card has some strange information on it, like a creation date of 1980. I assumed this would cause problems but it was actually fine. You then need to connect the pedal to your computer and get it in boot mode. It's very important that you have the correct power source attached while you do that. You then update the firmware and it should function normally after that. (Normally for the ditto X4, it should be said is **bleep**tily. There is still some strange behavior when recording a second loop. I sometimes get a weird blinking light. ) I would say if you've followed all of these instructions and you still can't get the pedal to connect to your computer, you may need to send it in for repair. 

    good luck!

    Caleb Heineman