Register |
Login
Shop for male chastity devices and other training tools through
this link to support us.
New stuff:
| << return to blog entries
2010-12-28 12:27:26 (15022 views) As you know if you've watched the video two posts below, we're back from the PCB assembly factory with pristine, immaculate, wonderful boards. ESD resistant, gold plated, oven-baked, pick-and-placed, in other words, just about perfect.
The story of how we managed to obtain results like these at a Chinese assembly plant where underpaid workers living and eating on site work with a massive sleep debt and terrible hand soldering equipment, is not for the weak of heart. We had to work day and night with the engineers. We had to sleep near the factory for two nights. We risked going home with nothing - no boards, and wasted components. We even had an emotional breakdown after two sleepless nights.
We are now home and feel like we've come back from the depths of hell.
Read on if you want the insider story!!!
We arrived at the factory in the morning, after an uneventful bus ride through the usual Chinese suburban areas.
We're used to the buses by now so we don't even bother trying to find seats that aren't broken - we immediately pick the cleanest, that is to say, the least dirty, and equipped with earplugs, which adequately shield from the incessant traffic noises, get ready for the bumpy ride.
The boss turned out to be somewhere else so we enthusiastically headed straight for the assembly room carrying our bag of precious components.
The first challenge we faced is the fact that pick and place machines require big reels of components, and have a waste of at least 100 pieces. We had been given a waste figure of 5 pieces per type, so we didn't arrive there with a surplus of parts. Fixing this required at least one hour as the engineers had to stitch component strips together with tape adding previously used strips to the ones we supplied.
The second challenge: we had asked the engineers what format they wanted the placement information in. Gerber files are normally used for pcb production and they do have position information for the components but they do not have a direction so we wanted to know what format they'd need to feed them into the machine. This turned out to be one of those questions that are too difficult to answer on the phone and so we arrived there with all the source files hoping to sort things out.
Unfortunately it turns out what they work based on is a PDF with a little arrow for the direction, which leaves the question open of how they figure out which direction the actual component has. For instance the LEDs have a T printed on the bottom, go figure which side of the T is the plus. We know how to find out, and they know too, but from finding out to not forgetting the direction while one walks from the desk to the machine is a whole different business, the business of precision, management, documentation and attention - the type of thing that's bound to get screwed up in China.
To get the data into the machine they fired up a program that lets the engineer define positions based on a picture file. We scanned the remote control PCB board directly and one guy started punching in numbers and defining the size of the panel and the PCB positions within the panel. We told him all the component stock numbers and directions one by one, this took another hour and a half. After which, the hard drive (a 2 year old Western Digital) completely died.
The staff were more interested in repairing the hard drive than going on with the job but it was already getting late so we took charge, took out the hard drive, installed it on another machine and tried to read it. Nothing - dead. Not the first time we see a western digital just die like that.
As we tried to fix the scanner the lower level engineering guys on the other side of the hall (basically some skinny teens in blue uniforms and bad haircuts with seemingly little background in what they were attempting) were feeding the radio chips into the machine's holders, unfortunately they left good part of the strips lying across the floor as they walked around, and kept walking dangerously close to the components so we were basically in panic trying to tell them to be careful, eventually they (lightly) stepped on the strip containing our chips so our polite requests turned into loud and firm admonitions and we took matters into our own hands as usual.
Anyway, the bad news with the PC failure was that we had to repeat the whole data entry process for the first board. The worse news was that the scanner could not be operated on another PC to scan the second board. Drivers were unavailable. Tried downloading the driver from Chinese sites, English sites, wouldn't work. After half an hour looking for drivers and downloading untold viruses and spyware from the blinking multicolored Chinese free driver download sites someone finally has the sense to bring in another scanner whose existence had been kept secret, so we scanned the second board and resumed data entry.
As we entered the data an engineer was trying to correctly put the solder on the PCBs using the steel stencils (first images in the video). He kept shorting the pads and trying again, then the manager came to do the job himself.
The next problem was that despite having already successfully tried pick-up and positioning of our radio module with the pick and place machine on our first visit, the engineers were now saying that because its bottom was not flat (?) the machine could not correctly position the module and it had to be put on by hand. By hand? We didn't come here to do stuff by hand guys... and, not flat? What's that about? Apparently some 0.1mm solder protrusions from the microvias on the bottom...
It turns out this "hand placing" is a very trivial deal, they brought in a lady with a very steady hand who put the chips on with great precision and speed, and the final result still looked totally professional. Hand placing is not the same as hand soldering. Essentially this one chip got positioned by hand but then got soldered in the oven with all the other chips. Same for the USB connector and some other stuff. The result: nice and clean.
As soon as the first remote control boards came out of the machine the battery connectors were soldered by hand (they're through hole so the machine doesn't place them), and we tested the new remotes successfully. More and more remote control boards came through. Only one was defective, but they manually fixed the pads on the smallest chip (the battery charging IC) and that fixed everything.
The second board was a lot more complex (the receiver). These boards came out of the machine looking perfectly neat and clean, but due to the miniaturization of our receiver there are some components that need to be hand-placed. Aside from the battery and motor connectors, we have a buzzer, and an LED and a capacitor that require proper lead length and to be mounted parallel to the PCB.
They set up an assembly chain with three different people putting these components in.
As soon as the first panel of four boards came out we tested them successfully.
However, after a while we found an odd board. As soon as we plugged in the battery the board started smoking! That's not good. We took out the battery immediately and analyzed the board for a while; eventually it turned out the solder protrusions under the module were shorting against the exposed vias on our PCB!
Things went downhill. Already tested boards started failing in different ways. On some the accelerometer wasn't working, some couldn't receive packets.
We kept checking and resoldering, and as one problem got fixed another one emerged. By this time we were into the second day and survivors of the most boring Chinese factory employee Christmas karaoke night, witnesses to the worst singer in China, the boss mr Wen - now no longer on the spot due to an engagement elsewhere.
Some boards started looking worse and worse after iterations of soldering and desoldering. Buzzer sound was uneven - some buzzers were barely audible.
Around 3am, when it became clear even the technical manager was clueless and sleepy and just basically touching each contact with a huge, oxidized soldering iron hoping things would improve, we had a nervous breakdown: we were desperate, feeling the concrete possibility we had just wasted all our components. We started complaining about their equipment (they try to solder 0.5mm pitch pads with a 5mm diameter oxidized tip, a filthy black soldering sponge, they have million dollar automatic machines but can't even buy a 10$ soldering iron...)
The manager got mad and started behaving and talking quite rudely but we absolutely couldn't let this turn sour because the job had started and all our components were on the boards.
We touched rock bottom when it became apparent that even good boards would smoke if you bent them lightly (the PCB is only 0.8mm thick) - this is how we learned that the contact between vias on the modules and vias on the PCB was the problem. We can't have the boards fail if they get twisted slightly, this can occur when you close the enclosure or due to thermal fluctuations. These were the hardest moments in two years of development, we were about to have a monster of an anti-climax with intolerable time and component losses. A worst case scenario included a 3 month lead time delay to get more radio parts.
We decided to desolder all the radio chips and add some tape below them to prevent shorting and resolder them. This took 24 hours, and our great luck was the availability of hot air desoldering equipment and a staff experienced in using it. This is what really saved us because we managed to move things around massively without ever damaging a radio module (we fried 3 or 4 accelerometers but that's minor).
We stayed there during the night shift until 3am, and again the morning after starting at 9am, finding it almost impossible to sleep with all our DL2000 boards in the hands of seemingly dexterous but easily distracted engineers with huge oxidized soldering irons tips.
Eventually all the chips were resoldered, and the smoking problem was resolved. We still had accelerometer failures and RF reception errors. It took a few hours but we tracked this down to improper soldering of the accelerometer, and the accelerometer sometimes frying as the hot air was being used to remove the radio module.
We fixed the defective items one by one, resoldering and replacing the accelerometers. Another problem surfaced. All previously tested boards now had LED failures. This was really hard to figure out. The LED pads had become disconnected from their outbound traces due to board bending when the LED was bent back and forth to expose the module for desoldering! The only way to fix this was to scratch the small trace under the microscope and use some crazy soldering technique to restore the contact between the trace and the pad (the engineer applies the iron and then switches off the iron's power supply, not sure how this works, but it accomplishes this very tricky microsoldering job that seems to go against surface forces acting on the solder which want to keep solder on the pad rather than bridge it to the trace, so we asked no further questions).
Almost all boards were repaired with the exception of two or three difficult cases. The main problem now was that the boards looked like they had been to hell and beyond and back. Flux residues and bent LED and capacitor leads really made them look like trash. This was already the Sunday after Xmas, and two engineers came to offer unpaid help even though it was their day off. Before we could become too appreciative though, one started chatting away with his girlfriend through his cell phone and the other started acting quite rudely due to our bashing of his soldering equipment.
We told them to clean the boards and restore the leads. Obviously our aesthetic ideals can appear a bit out of the ordinary to people who're used to total trash PCBs going inside non-transparent, non-diamond polished enclosures inside cheap made in China supermarket toys. They cleaned, and cleaned, and we had them continue until the boards were immaculate; then, we did the final cleaning ourselves to achieve perfection. One of the guys stopped chatting with his girlfriend and turned out to be somewhat of an artist at restoring component leads and giving them a good, youthful appearance. You wouldn't know these were bent back and forth a number of times.
At about 6pm the angry guy could no longer bear working for us and started wanting to leave. By this time we knew we could fix whatever new problems arose, even as the "already tested twice" boards started to malfunction. We knew we no longer needed them - we have better equipment at home and are able to solve any soldering issue with much greater care and concentration. So we left, not knowing exactly what to expect, but a bit heartened by the fact we now had almost all our boards once again working and - despite the resoldering - looking the way we want them to look.
We've been testing these boards for the past couple of days, terribly preoccupied that they may again cease functioning.
We have done some fixing since then, but we're proud to announce that all the boards we came home with are now working, and they have all survived three different comprehensive tests including accelerometer test, radio reception and range, etc. (aside from previous tests at the factory)
We will continue to test these boards until we ship. By now we're 99% sure there will be no more failures and the good thing is that we're now able to repair any malfunction in a couple of minutes so anything happening in the field right now would not be as big a deal as we had anticipated: we can receive a defective item and repair it easily rather than replacing it.
So at this time, after this terrible emotional roller coaster, we're now very proud of our guys and excited to show people what we've accomplished. Despite the trouble and the reworking the boards look just as good as when they came out of the machines (except the solder joints on the module are a bit bigger and not flat, nothing you would notice if you're not looking specifically for it).
The boards are well behaved and ready for use. They look beautiful. They are the result of a passion so great it has overcome any and all screw-ups China has thrown at us.
A terribly stressful Christmas with a happy ending, and we approach launch with confidence.
Comments
Post new comment
 |
By nuvaris at 2010-12-28 14:04:05 Reply 
